From chemical compounds to herbal interventions: a transfer learning-based perturbational transcriptome prediction framework for cancer drug discovery

OBJECTIVE: Transcriptomic perturbation profiles from tumor cell lines serve as the core molecular basis for cancer drug discovery and mechanism of action (MOA) analysis. Traditional Chinese medicine (TCM) holds great anticancer potential, yet the multi-component and multi-target properties pose major challenges for systematic mechanistic investigation. The scarcity of herbal intervention transcriptomic data severely restricts transcriptome-based anticancer TCM research, unlike widely available large-scale chemical compound perturbational datasets. This study aims to establish a predictive framework for herbal transcriptional responses in tumor cell models to address this critical data bottleneck. METHODS: A transfer learning-based encoder-decoder prediction framework integrated with a self-attention mechanism was developed. The model was pre-trained on large-scale connectivity map compound perturbation datasets with paired baseline transcriptomic profiles, then fine-tuned with limited herbal perturbation data covering 11 herbs across 4 tumor cell lines using a shared gene set as the molecular basis. RESULTS: = 0.8561, Pearson correlation coefficient = 0.9258), outperforming baseline models with robust generalization to unseen herbal interventions. Transfer learning markedly improved prediction accuracy and stability under data-limited conditions. CONCLUSIONS: screening, preliminary MOA exploration, and multi-herb prescription synergistic pattern analysis in cancer drug discovery.