ADMET Liability Flagger: Honest Single-Molecule Developability Triage#

A technical white paper

Abstract#

Early drug-discovery teams often optimize the variable that is easiest to see: potency against a target. Potency is necessary, but it is not sufficient for a molecule to become a useful probe, lead, or development candidate. Solubility, permeability, metabolic interaction, cardiac-ion-channel activity, mutagenicity, hepatotoxicity, physicochemical properties, structural alerts, and synthetic tractability can all change the value of a hit. When those dimensions arrive late, a program may discover that it has climbed the wrong hill: an apparently strong potency series has accumulated liabilities that are costly to understand or redesign.

The ADMET Liability Flagger is a proprietary v0 system for an earlier decision point. It accepts one molecule as a SMILES string and produces a structured, printable liability report card. The implementation combines deterministic RDKit rules with six offline-trained XGBoost models over Morgan fingerprints. Each predictive result includes the estimate, a configurable pass / caution / risk band, provenance, and an endpoint-specific applicability-domain signal based on maximum Tanimoto similarity to that model's fit-set chemistry. Low-similarity estimates are explicitly marked out of distribution. An optional Vertex AI layer can translate the structured evidence into a concise medicinal-chemistry narrative, but remains additive: missing cloud configuration or credentials never removes the numeric report.

A July 2026 evidence refresh reconstructed the original scaffold-held-out partitions from hash-matched local TDC data and re-scored all 6,331 held-out endpoint records with the committed boosters, independently of the stored metric values. All eight metric values reported by the manifest reproduced exactly. The four classifiers achieved a macro AUROC of 0.848410, the two regressors a macro R² of 0.497976, and the low-confidence applicability group carried 12.5% more endpoint-normalized absolute error than the non-low group. These remain retrospective baseline results, not prospective or wet-lab validation, but they provide an implementation-level check that the artifact bundle and uncertainty policy operate on the evidence claimed by this paper.

The system's central engineering thesis is honest uncertainty at the point of triage. It does not collapse heterogeneous endpoints into a single score, present uncalibrated classifier outputs as clinical probabilities, or treat a structural filter as experimental proof. Models are trained offline, committed in a validated artifact bundle, and loaded without a TDC runtime dependency. A pure per-molecule profiling seam lets a future hit-list triage and re-ranking layer wrap the same core without rewriting it. This paper describes the problem framing, architecture, model methodology, confidence policy, interpretation boundary, deployment design, measured baseline results, quality controls, limitations, and roadmap.

Table of Contents#

1. Introduction & Motivation
2. Design Goals, Constraints & Anti-Goals
3. System Architecture
4. Molecule Ingestion & Deterministic Rule Layer
5. Predictive Model Methodology
6. Applicability Domain: The Honesty Core
7. Bands, Provenance & Report Semantics
8. Optional Vertex Interpretation
9. API Contract & Reusable Profiling Seam
10. Scientific User Interface
11. Single-Container Deployment
12. Baseline Evaluation
13. Correctness & Quality Assurance
14. Security & Supply-Chain Posture
15. Limitations & Future Work
16. Conclusion
17. Appendices

• References

1. Introduction & Motivation#

1.1 The wrong-variable problem#

Discovery programs need measurable objectives. Potency is especially attractive because it can be ranked, optimized, and communicated as a simple number. That simplicity creates a behavioral risk: the program can begin treating potency as the objective rather than one constraint in a multidimensional optimization problem.

Developability liabilities are less tidy. A molecule can be potent and poorly soluble. It can cross an epithelial model but inhibit a major metabolic enzyme. A safety signal may be real, chemistry-specific, assay-specific, or merely outside the model's experience. Structural filters can identify patterns worth examining while also producing false positives. Synthetic accessibility can affect cycle time without determining whether a route is actually practical. These dimensions are heterogeneous, and their uncertainty is not interchangeable.

The useful intervention is therefore not “predict the fate of the molecule.” That claim would exceed what baseline public-data models can support. The useful intervention is narrower: make plausible liabilities and uncertainty visible early enough to change the next decision. A report that prompts a solubility measurement, flags a low-confidence hERG estimate for follow-up, or prevents a structural alert from being mistaken for assay evidence can be valuable without being definitive.

1.2 Why a single-molecule profiler#

Single-molecule profiling matches an important unit of work: a chemist or project team has one structure under active consideration and needs a reviewable evidence surface. The input is intentionally limited to one typed or pasted SMILES. The output is intentionally explicit: descriptors, rule outcomes, six endpoint estimates, confidence labels, and optional interpretation.

This scope has three benefits. First, it keeps error handling comprehensible: the service either parses one molecule or returns one clean structure error. Second, it supports evidence review rather than hiding assumptions behind a ranking. Third, it establishes a stable atomic operation for later batch use. A hit-list product can call the same operation repeatedly; it does not require the v0 profiler to pretend that a portfolio-ranking policy already exists.

1.3 The core idea: separate signal, policy, and confidence#

The report treats three questions as independent:

• What did the estimator return? A continuous value or positive-class score.
• How does the current triage policy label it? Pass, caution, or risk under visible thresholds.
• How familiar is the chemistry to that endpoint's model? High, medium/edge, or low/OOD under visible Tanimoto boundaries.

A risk band does not imply high confidence, and high confidence does not imply a pass. The system displays both axes because collapsing them would discard decision-relevant information.

This separation is the product's trust mechanism. It is also an architectural choice: raw predictions, threshold policies, applicability-domain references, and presentation models are distinct objects rather than a single opaque score.

2. Design Goals, Constraints & Anti-Goals#

2.1 Success criteria#

A reviewer should be able to:

1. submit a valid SMILES and receive the full profile without any cloud credentials;
2. understand which rules and thresholds produced each band;
3. see when an ML estimate is outside its endpoint's training neighborhood;
4. inspect model dataset, release, date, semantics, and held-out metric;
5. print a complete one-page-oriented evidence card without expanding screen accordions;
6. enable a structured Vertex narrative without introducing API keys;
7. replace a reviewed baseline bundle without changing the /profile response shape; and
8. reuse the pure per-molecule core in a later hit-list workflow.

2.2 Hard constraints#

2.3 Anti-goals#

The v0 is not a QSAR platform, electronic lab notebook, compound registry, assay database, route planner, clinical risk calculator, or autonomous design agent. It does not produce an overall “developability score.” It does not claim mechanistic causality from fingerprint models. It does not ask an LLM to invent missing measurements. It does not silently accept partial or smoke-trained model bundles.

2.4 Maturity#

This is a runnable baseline MVP. Production-oriented mechanisms include strict artifact validation, typed response contracts, clean error envelopes, non-root container execution, optional-cloud isolation, responsive and print UI behavior, and automated backend and frontend tests. Its scientific maturity is narrower: the models are public-data baselines evaluated on one scaffold split, the classifiers are uncalibrated, and no prospective or external validation has been performed.

3. System Architecture#

The system is a staged pipeline with a small typed boundary between stages.

3.1 Module map#

3.2 The pure profiling seam#

profile_molecule(...) accepts an RDKit molecule, an already-loaded ModelBundle, optional thresholds, and an applicability policy. It creates one query fingerprint, applies all six boosters, assesses all six endpoint domains, and returns typed ML results. It performs no filesystem, environment, network, or cloud I/O.

profile_one_molecule(...) combines those results with the deterministic rule layer, a bounded best-effort display conformer, counts across the six ML bands, warnings, and a scope caveat. It is also free of cloud I/O. The API adapter handles cached bundle loading and optional interpretation. A future batch service can therefore load dependencies once and loop over molecules without recreating HTTP or Vertex behavior inside the scientific core.

4. Molecule Ingestion & Deterministic Rule Layer#

4.1 Defensive ingestion#

The parser accepts text, trims whitespace, rejects empty input, limits the string to 4,096 characters, and asks RDKit to parse with sanitization enabled. Native RDKit parser diagnostics are suppressed so clients receive a single stable error rather than process-level chemistry logs. A valid molecule is returned with the submitted text, canonical isomeric SMILES, and RDKit object.

The molecule summary exposes formula, total and heavy atom counts, disconnected-fragment count, and formal charge. Multiple fragments are not silently removed; the report warns that descriptors and models reflect the submitted multi-fragment structure. This is honest but intentionally conservative. Salt stripping and parent selection are future policy decisions because automatic normalization can change the chemical question.

4.2 Descriptors and medicinal-chemistry heuristics#

The deterministic panel calculates molecular weight, Crippen logP, topological polar surface area, hydrogen-bond donors and acceptors, and rotatable bonds. These values feed transparent Lipinski and Veber checks. One Lipinski or Veber violation receives a caution band; multiple violations receive a risk band. That banding is a product policy over heuristics, not a claim that a molecule with one violation is unusable.

Synthetic accessibility uses RDKit's contributed Ertl-style scorer on an approximate 1–10 scale, with lower values treated as easier. The v0 bands scores at <= 4, > 4 to <= 6, and > 6. The score is a complexity heuristic; it does not know available building blocks, proprietary routes, protecting-group strategy, yield, cost, or a laboratory's capabilities.

4.3 Structural alerts with lightweight attribution#

The PAINS catalog is queried through RDKit's filter framework. Each match can report its catalog name, family, scope/description, matched SMARTS, and zero-based atom indices. This is enough to make the alert inspectable and establishes a seam for later structure highlighting.

PAINS is treated as a review trigger, not an assay verdict. A match may justify checking interference, reactivity, aggregation, or literature context; absence of a match does not establish clean pharmacology or safety. The UI repeats that caveat where alerts are displayed.

5. Predictive Model Methodology#

5.1 Endpoint and target semantics#

The offline trainer draws six single-prediction datasets from the Therapeutics Data Commons. The v0 uses two regressions and four binary classifiers.

The classification values are described throughout the API and UI as uncalibrated positive-class probability estimates. AUROC measures ranking discrimination on the held-out data; it does not make the numeric output calibrated for decision costs or prevalence in a new program.

5.2 Feature contract and scaffold split#

Training and serving share one serialized feature specification: an achiral Morgan bit vector generated by RDKit with radius 2 and 2,048 bits. The query is featurized once, then passed to all six models and applicability-domain assessment. Achirality is a deliberate baseline simplification recorded in the manifest, not an accidental omission.

Each cleaned dataset receives a deterministic molecular-scaffold split: 70% train, 10% validation, and 20% test. Train and validation are combined for the final fit; the test partition remains held out for the reported metrics. Scaffold splitting is more demanding than a naive random split for novel chemotypes, but a single split is still only one estimate of generalization.

The XGBoost baseline uses histogram tree construction, 300 estimators, depth 6, learning rate 0.05, row and column subsampling of 0.8, L2 regularization, and a fixed seed. Classifiers use binary:logistic; regressors use reg:squarederror. Evaluation uses scikit-learn's standard ROC AUC, mean-squared-error, and R² metrics.

5.3 Offline-only training and committed artifacts#

scripts/train_models.py imports PyTDC lazily, cleans structures and labels, builds fingerprints, fits estimators, evaluates held-out data, and writes artifacts. TDC and pandas live in requirements-train.txt; the serving path does not depend on either package.

Each endpoint produces:

1. native XGBoost JSON for stable model I/O rather than Python pickle;
2. a metadata sidecar containing task semantics, feature specification, split, metric, training counts, dependency versions, estimator parameters, and data digest; and
3. a compressed, pickle-free NPZ of packed train+validation fingerprints and bit counts.

The root manifest orders all six endpoints and records exact byte sizes and SHA-256 hashes. Before serving predictions, the runtime rejects capped, partial, incompatible, or corrupted bundles and validates sidecar equality, feature count, objective, safe paths, fingerprint dimensions, bit counts, and shared release metadata. In this manifest, production_ready=true means that the bundle is complete, uncapped, and compatible with the strict loader. It does not mean that the models are scientifically, clinically, or regulatorily validated for production decisions. XGBoost's documented save_model JSON path provides the model format; the project manifest supplies the scientific and integrity context that the booster alone lacks.

6. Applicability Domain: The Honesty Core#

6.1 Exact nearest-neighbor signal#

For each endpoint, the trainer commits the Morgan fingerprints of that endpoint's train+validation fit set. At inference, the query fingerprint is packed to 256 bytes. Vectorized bitwise intersection and precomputed bit counts yield the exact Tanimoto similarity to every reference:

The maximum similarity, denoted $s$, becomes the endpoint's applicability signal. No approximate index or sampling is used in v0. The reference set remains packed in memory, so exact search is practical at the current dataset sizes.

6.2 Confidence policy#

The thresholds are configurable and returned in every response. They are operational boundaries, not confidence intervals and not probabilities that a prediction is correct. A value of 0.46 does not create a discontinuous scientific truth compared with 0.44; the labels create an understandable triage policy over a continuous similarity signal.

6.3 Why confidence is endpoint-specific#

The six datasets contain different compounds and different numbers of fit examples. A molecule may be familiar to the Ames model and unfamiliar to the DILI model. Reusing a single global chemistry reference would obscure that difference, so every endpoint owns its fit-set reference and reference count.

This design also prevents a misleading shortcut: a molecule should not inherit “high confidence” merely because it resembles chemistry from an unrelated endpoint. The domain signal answers a precise question, has this model seen nearby fingerprint chemistry in its own fit data?, not the broader question of whether the molecule is drug-like.

6.4 What the signal does and does not establish#

Max Tanimoto is interpretable and reproducible, but it captures only similarity under the chosen fingerprint. It does not quantify label noise, local target smoothness, training density beyond the closest point, assay-domain mismatch, conformational effects, stereochemical effects omitted by the achiral representation, or model variance. A close neighbor can still have a different label; a distant molecule can still be predicted well.

The v0 therefore calls the result an applicability-domain or confidence flag, not a statistical uncertainty interval. Its value is behavioral: it prevents the interface from showing every model number with identical visual authority. The reproduced held-out analysis in Section 12.3 found higher pooled error behind the low-confidence flag, with one endpoint-level exception. That evidence supports visible qualification; it does not turn Tanimoto similarity into a calibrated probability of correctness.

7. Bands, Provenance & Report Semantics#

Banding is applied after prediction through runtime configuration. Default regression policies mark solubility as caution at <= -3 and risk at <= -4 log10(mol/L), and Caco-2 as caution at <= -5.15 and risk at <= -6 log10(cm/s). The four classifier endpoints share caution at >= 0.30 and risk at >= 0.60.

Those thresholds are directional review policies. They are not optimized decision thresholds for a specific assay protocol, target product profile, species, route, or indication. Because thresholds are returned with their source and direction, teams can change policy without retraining and still audit how a band was produced.

Each endpoint also exposes its dataset name, model release, training date, prediction semantics, and held-out metrics. Rather than combining six heterogeneous predictions into an arithmetic score, the report presents distribution counts and lets users inspect each result. This is less superficially convenient, but avoids equating, for example, a solubility regression residual with a mutagenicity classifier score.

8. Optional Vertex Interpretation#

The interpretation layer addresses a communication problem rather than a missing prediction problem. A structured profile can be cognitively dense; a careful narrative can identify the few liabilities that merit attention together and propose a sequence of de-risking work. It must not become the source of scientific facts.

The adapter sends a constrained evidence subset: molecule summary, descriptors, rule outcomes, predictions, bands, applicability objects, structural alerts, warnings, and the scope caveat. The system instruction requires a distinction between model signals and structural alerts, explicit treatment of OOD results, no invented thresholds or atom attribution, and no composite score. Generation uses a JSON response schema, low temperature, a configurable default output ceiling of 4,096 tokens, strict Pydantic validation, sorted priorities, and retries for request failures or malformed output. Usage logs include the model, prompt version, token counts, finish reason, and response length when the SDK exposes them. If retry exhaustion ends with a MAX_TOKENS finish reason, the result is diagnosed as truncation rather than generic malformed JSON.

Authentication is centralized through Application Default Credentials. Local development uses gcloud auth application-default login; Cloud Run uses the attached service account. There are no API keys. For development, Pydantic Settings reads a Git-ignored local .env case-insensitively; Cloud Run supplies deployment values as environment variables. Configuration values default to explicit PLACEHOLDER_* strings and are validated only when Vertex is enabled.

Three states are first-class:

• generated: validated structured narrative and model provenance;
• skipped: disabled or not configured, with the numeric report complete; and
• failed: initialization or generation did not succeed, again with the numeric report complete.

The design treats an LLM as a fallible interpreter of trusted structured evidence. It does not let the LLM change endpoint values, bands, confidence labels, or rule matches.

9. API Contract & Reusable Profiling Seam#

POST /profile accepts { "smiles": "..." } and returns a versioned ProfileResponse. Invalid chemistry returns HTTP 400 with a stable code, message, and smiles field. Invalid request shape returns HTTP 422. Model-bundle failures are sanitized as HTTP 503 without exposing local paths; unexpected failures become a generic HTTP 500 response while detailed exceptions remain in service logs.

The response contains molecule and descriptor summaries, a discriminated conformer_3d result, deterministic rule results, structural alerts, ordered predictions, summary counts, interpretation state, warnings, a scope caveat, and generation and model metadata. A generated conformer includes bounded atom coordinates, bonds, force-field provenance, and a scientific caveat; an unavailable result explains the non-blocking reason. Pydantic models define the internal and external contracts. Frontend TypeScript types mirror and defensively validate the public response before geometry reaches WebGL.

The API caches the validated ModelBundle on first use. It does not cache molecule responses. This keeps the scientific operation reusable and avoids hidden cross-request state. A future batch endpoint can parse a list, collect input-level errors, and call profile_one_molecule for each valid structure while retaining the same result model.

10. Scientific User Interface#

The React 18 SPA is designed as an evidence surface rather than a marketing dashboard. Deep teal is the primary accent; pure red is reserved for risk. Band badges and confidence badges are independent and have textual labels. Endpoint accordions expose threshold policy, nearest Tanimoto, applicability boundaries, positive-class semantics, dataset, model release, training date, and metrics. A compact molecular SVG favicon carries the same teal, carbon, nitrogen, oxygen, and sulfur-inspired visual language into the browser tab.

The visual overview uses Recharts for counts, not a composite gauge. The report includes generated/skipped/failed narrative states and a persistent scope caveat. Warnings are deliberately placed after the evidence sections, both on screen and in the print tree, so they remain prominent without interrupting the result hierarchy. A dedicated print tree renders every endpoint and rule even when screen accordions are closed. It combines a 10 mm A4 page margin with 6 mm top and 8 mm side/bottom internal padding, retaining a readable gutter even when the browser print dialog is set to no margins. Dark and light themes use CSS tokens, focus states are visible, status meaning is not color-only, and reduced-motion settings remove report and accordion motion.

The landing hero retains one GPU-light decorative react-three-fiber motif. The full report is code-split until a profile exists, and the Three.js canvas is separately lazy-loaded. After a valid response, the report header renders the submitted molecule as an instanced ball-and-stick model with persistent per-instance, CPK-inspired element colors. Coordinates are centered and bounded before rendering. The generated structure does not auto-rotate; it moves only when the user drags it. An intersection observer avoids mounting the WebGL canvas until it is near the viewport, and the hero context is released offscreen. Unavailable WebGL receives a molecule-specific projected SVG fallback, while unsafe or failed conformer generation receives a truthful text state. Endpoint evidence and the printable report remain two-dimensional.

The on-screen geometry is one deterministic computational display conformer generated by RDKit ETKDGv3, with MMFF94s or UFF optimization when parameters are available. Hydrogens participate in conformer generation but are omitted from the display payload to reduce transfer and rendering work. The conformer is not measured, not a global-minimum claim, not a bioactive pose, and not a conformer ensemble. Disconnected fragments are deliberately not assigned arbitrary relative positions. The display conformer is excluded from the Vertex prompt and is not an input to any ADMET model.

11. Single-Container Deployment#

The Dockerfile has two stages. A Node 20 stage runs npm ci and vite build. A Python 3.11 slim stage installs pinned runtime dependencies plus Linux libraries for RDKit rendering and XGBoost OpenMP, copies application code and committed models, and copies only the compiled SPA from the Node stage. The final process runs as UID 10001 and listens on $PORT through Uvicorn.

API routes are registered before production static handling. /assets receives strict static-file behavior; unmatched GET routes return index.html for client-side routing. /health and /profile retain API semantics, including a 405 for the wrong /profile method. If no compiled frontend exists in local API development, FastAPI remains available in API-only mode.

Cloud Run supplies the ingress port and, when configured, the runtime service-account identity. The numeric service requires no Vertex role; enabling interpretation requires the Vertex AI API and roles/aiplatform.user on the attached runtime identity. The recommended configuration uses a dedicated user-managed identity such as admet-cloud-run, with no service-account key or GOOGLE_APPLICATION_CREDENTIALS override; Cloud Run supplies ADC. With repository CI/CD, build and deployment authority belongs to the build identity, which receives narrowly scoped permission to act as the runtime account, not to the application runtime. This separation avoids granting Cloud Build or Cloud Run administration privileges to code serving public requests. The documented image build targets linux/amd64, consistent with the Cloud Run container contract.

12. Baseline Evaluation#

12.1 Reproduced held-out results#

The committed manifest's production_ready=true state means complete, uncapped, and loader-compatible rather than scientifically validated for production use. The results use one deterministic TDC scaffold split; fit counts combine train and validation. Counts are retained dataset records after cleaning: the trainer canonicalizes structures but does not deduplicate them, so these are not asserted to be unique molecules.

For the revision 1.1 evidence refresh, the six local TDC source tables were cleaned, canonicalized, and split again with the recorded seed and training code. Each reconstructed dataset digest matched its manifest digest. The committed boosters then re-scored all 6,331 endpoint test records (not necessarily distinct molecules across datasets), reproducing every reported AUROC, RMSE, and R² value exactly to six decimal places. This verifies the local data-to-split-to-artifact evaluation path; it is not an independent external dataset.

12.2 Baseline-relative summary metrics#

The aggregate values below provide concise research metrics while keeping their comparison baselines explicit. They are macro averages that weight endpoints equally; they are not improvements over a prior product release or a medicinal-chemist workflow.

The normalized classifier value is:

It describes ranking lift over chance, not calibrated probability accuracy. For regression, $\mathrm{R}^2 = 1 - \mathrm{SSE}/\mathrm{SST}$; the unweighted macro value is:

The macro R² can therefore be read as the equal-weight mean of the endpoint-wise squared-error reductions against each endpoint's held-out-mean reference. Separately, all six runtime estimates carry an exact endpoint-specific domain assessment; one profile searches 25,297 endpoint-reference records in total.

12.3 Does the low-confidence flag locate harder predictions?#

The same reconstructed test partitions were used for a retrospective applicability check. For endpoint e and test record i, absolute error a and normalized error z were:

This divides each error by its endpoint's overall held-out mean absolute error so regression and classification records can be pooled without mixing their native scales. The pooled statistic is record-weighted after this within-endpoint normalization. Classification error here is descriptive absolute score error against the binary label; because scores are uncalibrated, this is not a calibration analysis.

The pooled enrichment ratio E is:

Pooled low-confidence predictions had mean endpoint-normalized absolute error 1.1132, compared with 0.9894 for non-low predictions: a 12.5% enrichment of error behind the warning. A 10,000-resample observation bootstrap (seed 1729) produced a percentile ratio 95% interval of 1.046–1.206. The direction held for five of six endpoints; CYP3A4 was the explicit counterexample. The result supports the interface decision to qualify low-similarity estimates, but it does not establish a universally monotonic error model, an endpoint-specific operating threshold, or prospective validity. The bootstrap is descriptive and does not model scaffold or dataset dependence.

12.4 Local runtime characterization#

A local single-process benchmark compared the implemented shared/cached path with deliberately naive ablations while confirming identical predictions and nearest-neighbor similarities across ten fixed drug-like molecules. Vertex and network latency were excluded.

The warm request p95 was 32.718 ms, and all 80 cached requests completed below 100 ms. The benchmark ran through FastAPI TestClient on macOS ARM64 with Python 3.11.13, RDKit 2023.09.6, and XGBoost 2.1.4. It is an implementation benchmark, not Cloud Run end-user latency: it excludes container startup, network transit, browser rendering, and Vertex generation. The reload baseline also benefited from the operating-system file cache. The small difference at the full ML-core level shows that exact applicability searches, not fingerprint construction, dominate that stage; the larger request-level gain comes from validating and loading the model bundle once per process.

12.5 Interpretation of the evidence#

The classification AUROCs show useful rank discrimination on these held-out scaffold partitions. They do not establish calibration, sensitivity at a chosen operating point, prospective performance, or transfer to a proprietary chemistry series. Regression R² values around 0.5 indicate that the models explain a meaningful but incomplete share of held-out variation; the solubility RMSE in particular is large enough to require substantial caution for molecule-level use.

Dataset size matters. DILI and hERG have much smaller test and fit sets than CYP3A4 or Ames, which increases uncertainty in both model behavior and the applicability reference. Dataset labels also inherit assay definitions and curation choices. No endpoint-level metric confidence intervals are reported, and no claim is made that one split captures all plausible scaffold distributions. The pooled applicability bootstrap interval does not substitute for those missing endpoint-level intervals.

The proper reading is therefore “baseline directional evidence with visible domain limits.” The models demonstrate an end-to-end, upgradeable system and can prioritize experiments. The aggregate metrics, error-enrichment result, and local latency benchmark do not satisfy the evidence bar for replacing experiments or claiming a measured business outcome.

13. Correctness & Quality Assurance#

The implementation has 55 backend tests and 35 frontend tests; the revision 1.1 refresh reran all 90 successfully. Backend coverage includes SMILES errors; deterministic conformer generation and fallbacks; molecule summaries; rule behavior; feature dimensions; deterministic training components; strict manifest and artifact validation; objective and feature compatibility; exact applicability calculations; threshold boundaries; API errors; Vertex parsing, retry, and truncation states; and SPA routing. An autouse fixture prevents a developer's root .env from changing test behavior. Frontend coverage includes strict conformer payload validation, bounded molecule geometry, input-derived viewer fallbacks and motion policy, input flow, themes, separate band and confidence rendering, print completeness, footer warning order, structural attribution, interpretation states, and accessible animated disclosure.

A targeted additive-LLM fault exercise covered five molecules under five interpretation conditions: disabled, placeholder configuration, client/ADC initialization failure, invalid structured output after retries, and maximum-token truncation. All 25 cases retained a complete numeric report, including 150 of 150 endpoint predictions and 100 of 100 deterministic rule results. These were injected deterministic failures rather than a live Google Cloud outage, so the result validates the application boundary, not external service availability.

The held-out error analysis, local microbenchmark, and five-condition fault matrix are ad hoc revision 1.1 evidence-refresh exercises rather than committed recurring test targets or part of the 90-test suite. Their methods, sample sizes, and limitations are recorded here so the claims remain reviewable; promoting them into versioned evaluation scripts and release gates is future quality-system work.

Ruff enforces the Python codebase; ESLint and TypeScript check the SPA. Production builds run through Vite. The multi-stage Docker image has been built and smoke-tested as a non-root single service, including root SPA delivery, nested client routing, /health, and a real six-endpoint aspirin profile.

Quality gates reduce implementation risk, not scientific risk. Unit tests can prove that a Tanimoto threshold is applied correctly; they cannot prove that the threshold is biologically optimal. The project treats those as different validation questions.

14. Security & Supply-Chain Posture#

Python package dependencies are pinned. The committed manifest records the exact Python, NumPy, RDKit, scikit-learn, XGBoost, and PyTDC versions used for training. The final container excludes raw TDC tables, training-only dependencies, tests, documents, local environments, and credential-shaped files. It intentionally includes the packed fit-set fingerprints required for applicability assessment, runs as a non-root user, and contains no secrets. The Node build and Python runtime base tags are versioned, but image digests and operating-system package versions are not fully pinned.

Application Default Credentials avoids embedded keys, but IAM still requires least-privilege configuration. A dedicated Cloud Run runtime service account should hold only the API permissions the application needs, currently Vertex AI User when interpretation is enabled, while a separate build/deploy identity owns CI/CD permissions and narrowly scoped authority to attach that runtime identity. The default Compute Engine service account can function, but its shared role history makes it a weaker production boundary. Public Cloud Run deployment is an explicit choice and should be paired with rate limiting, monitoring, and organizational controls before sensitive use. Artifact SHA-256 checks detect modification but do not sign provenance; a mature release process should add image digests, vulnerability management, artifact signing/attestation, and controlled promotion.

15. Limitations & Future Work#

15.1 Scientific limitations#

• Baseline data and one split. Results derive from public TDC datasets and one scaffold split. They are not externally or prospectively validated on the intended chemistry.
• No deduplication or leakage audit. Training canonicalizes structures but does not deduplicate retained records or independently audit cross-partition leakage beyond the deterministic scaffold split.
• Uncalibrated classifiers. The four scores rank the positive class but are not calibrated probabilities. Thresholds are common triage policy, not endpoint-specific decision analysis.
• Fingerprint representation. Radius-2 achiral bits omit explicit 3D conformation, assay context, dose, species, route, formulation, stereochemical distinctions, and learned molecular representations.
• Display conformer only. The generated 3D structure is a deterministic visualization aid. It does not resolve protonation, tautomer, solvent, protein, pH, or ensemble context, and it is not used by the radius-2 fingerprint models.
• Label and assay heterogeneity. Public targets may combine protocols or curation decisions that differ from a user's assays. Model confidence cannot repair label mismatch.
• Nearest-neighbor domain only. Maximum Tanimoto does not model local error, density, epistemic variance, label cliffs, or uncertainty in the reference data.
• Retrospective applicability evidence. The 12.5% pooled error enrichment uses the same held-out datasets that characterize the baseline models, an observation-level bootstrap, and one fixed policy. It is encouraging diagnostic evidence, not external validation or a calibrated error guarantee; CYP3A4 moved in the opposite direction.
• Rules are heuristics. Lipinski, Veber, PAINS, and SA flags aid review but cannot determine oral exposure, interference, toxicity, or synthetic feasibility.
• No parent normalization. Multi-fragment inputs are warned about but not salt-stripped or standardized to a parent form. Tautomer and charge-state policy is limited to RDKit parsing/canonicalization.
• Limited attribution. PAINS offers matched atoms; ML models do not yet provide atom-level explanation. Fingerprint importance alone would require careful handling to avoid causal overinterpretation.
• Endpoint coverage. v0 excludes clearance, plasma-protein binding, volume of distribution, transporter interactions, broader CYP panels, species-specific endpoints, and many safety assays.
• No experimental loop. The service does not ingest laboratory results, learn from project data, or track whether a flagged liability was confirmed.

15.2 Product and operational limitations#

The service accepts one molecule per request and maintains no persistence, user identity, tenancy boundary, audit log, saved report, or portfolio state. Public deployment has no built-in rate limit. The model bundle loads per process, so scale-out instances repeat validation and memory use. Exact Tanimoto search is efficient at current reference sizes but will need profiling or indexing for much larger proprietary sets. The local latency figures in Section 12.4 exclude cold starts, Cloud Run scheduling, network transit, browser work, and Vertex generation; they must not be presented as hosted-service latency.

The Vertex narrative is constrained but remains generative output. It can summarize supplied evidence; it cannot establish mechanism, select an assay with knowledge of local capabilities, or guarantee that a suggested structural change preserves potency. Human review remains necessary.

15.3 Roadmap#

The next product layer is comparative hit-list triage and re-ranking. It should wrap the existing pure profiler, preserve per-molecule errors, and expose raw value, band, and confidence independently. Candidate ranking can then support configurable hard gates, endpoint weights, Pareto fronts, uncertainty penalties, and shortlist-only interpretation. The design should resist turning those weights into an unexplained universal score.

Model work should include repeated scaffold splits or nested evaluation, calibration analysis, confidence intervals, endpoint-specific operating thresholds, external/prospective sets, stronger representations, hyperparameter selection isolated from the held-out test, and documented label harmonization. Applicability work can compare nearest-neighbor similarity with density, conformal methods, ensembles, or local residual models while retaining an understandable user-facing explanation.

Chemistry work should add explicit standardization policy, optional parent selection, broader structural-alert catalogs, atom-highlight rendering, and carefully validated ML attribution. Platform work should add authenticated private deployment, request controls, observability, signed artifacts, model registry/promotion, drift review, and project-level experimental feedback. Each addition should preserve the v0 principle: uncertainty must remain visible rather than being averaged away.

16. Conclusion#

The ADMET Liability Flagger demonstrates a practical middle ground between a rule checklist and an opaque prediction service. It assembles deterministic chemistry, six offline baselines, explicit threshold policy, endpoint-specific applicability evidence, and optional structured interpretation into one reviewable artifact. The system can run without cloud dependencies, validates its committed scientific artifacts, and exposes a pure core for later batch orchestration.

Its most important feature is not any individual model metric. It is the refusal to present every number with the same certainty. By keeping estimate, liability band, and chemical-domain confidence separate, and by treating the LLM as optional interpretation rather than authority, the v0 provides a credible foundation for earlier developability conversations. The appropriate next step is not to overstate the baseline, but to validate, calibrate, expand, and connect it to comparative decision workflows.

17. Appendices#

Appendix A: Public service surface#

GET  /health     Lightweight liveness; does not load models or Vertex
POST /profile    One SMILES -> rules, predictions, confidence, optional narrative
GET  /            Compiled React SPA in production
GET  /assets/*    Hashed Vite assets in production
GET  /*            SPA fallback for client routes, after API/static routes

The minimum profile request is:

{
  "smiles": "CC(=O)Oc1ccccc1C(=O)O"
}

Appendix B: Confidence labels always surfaced#

Appendix C: Reproducibility commands#

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-dev.txt
python -m pytest
python -m ruff check app scripts tests

cd frontend
npm ci
npm run lint
npm test
npm run build

Offline retraining requires requirements-train.txt and network access to TDC. A full candidate should be written to a new directory and validated through load_model_bundle(...) before replacing all 19 tracked bundle files together.

Appendix D: Glossary#

• ADC: Application Default Credentials; the credential-discovery mechanism used by Google Cloud client libraries.
• ADMET: Absorption, distribution, metabolism, excretion, and toxicity.
• Applicability domain: The region in which a model is considered sufficiently supported by its reference chemistry under a defined method.
• AUROC: Area under the receiver operating characteristic curve; a threshold-independent ranking metric, not calibration.
• Caco-2: An epithelial cell model commonly used as an in-vitro permeability proxy.
• DILI: Drug-induced liver injury.
• Fit set: Train plus validation retained records used for final model fitting and v0 applicability references.
• Morgan fingerprint: A circular substructure fingerprint; v0 uses radius 2, 2,048 bits, without chirality.
• OOD: Out of distribution under the configured max-Tanimoto rule.
• PAINS: Pan-assay interference patterns; structural review alerts, not proof of interference.
• RMSE: Root mean squared error, in the target's reported units.
• SA score: Synthetic-accessibility heuristic on an approximate 1–10 scale, lower treated as easier.
• TDC: Therapeutics Data Commons.
• Tanimoto similarity: Intersection-over-union similarity for binary fingerprints.

References#

All URLs accessed July 2026.

1. Huang, K. et al. Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development. arXiv:2102.09548.
2. RDKit contributors. The RDKit Book and API documentation. rdkit.org/docs.
3. Rogers, D. and Hahn, M. Extended-Connectivity Fingerprints. Journal of Chemical Information and Modeling 50(5), 2010. DOI: 10.1021/ci100050t.
4. Chen, T. and Guestrin, C. XGBoost: A Scalable Tree Boosting System. KDD 2016. arXiv:1603.02754.
5. XGBoost contributors. Introduction to Model IO. xgboost.readthedocs.io.
6. scikit-learn contributors. Metrics and scoring: quantifying the quality of predictions. scikit-learn.org.
7. Google Cloud. Set up Application Default Credentials. docs.cloud.google.com.
8. Google Cloud. Cloud Run container runtime contract. docs.cloud.google.com.
9. Lipinski, C. A. et al. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced Drug Delivery Reviews 46, 2001. DOI: 10.1016/S0169-409X(00)00129-0.
10. Veber, D. F. et al. Molecular properties that influence the oral bioavailability of drug candidates. Journal of Medicinal Chemistry 45(12), 2002. DOI: 10.1021/jm020017n.
11. Baell, J. B. and Holloway, G. A. New substructure filters for removal of pan assay interference compounds (PAINS) from screening libraries and for their exclusion in bioassays. Journal of Medicinal Chemistry 53(7), 2010. DOI: 10.1021/jm901137j.
12. Ertl, P. and Schuffenhauer, A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics 1, 2009. DOI: 10.1186/1758-2946-1-8.
13. Google Cloud. Introduction to Cloud Run service identity. docs.cloud.google.com.