An explainable deep learning model using a chemical substructure-based approach for the exploration of chemical compound libraries identified structural classes of compounds with antibiotic activity and low toxicity.