Large-scale comparison of machine learning methods for drug target prediction on ChEMBL


Deep learning is currently the most successful machine learning technique in a wide range of application areas and has recently been applied successfully in drug discovery research to predict potential drug targets and to screen for active molecules.

However, due to
(1) the lack of large-scale studies,
(2) the compound series bias that is characteristic of drug discovery datasets and
(3) the hyperparameter selection bias that comes with the high number of potential deep learning architectures,
it remains unclear whether deep learning can indeed outperform existing computational methods in drug discovery tasks.

We therefore assessed the performance of several deep learning methods on a large-scale drug discovery dataset and compared the results with those of other machine learning and target prediction methods. To avoid potential biases from hyperparameter selection or compound series, we used a nested cluster-cross-validation strategy.

We found
(1) that deep learning methods significantly outperform all competing methods and
(2) that the predictive performance of deep learning is in many cases comparable to that of tests performed in wet labs (i.e., in vitro assays).


  author="Mayr, Andreas and Klambauer, G{\"u}nter and Unterthiner, Thomas and Steijaert, Marvin and Wegner, J{\"o}rg K. and Ceulemans, Hugo and Clevert, Djork-Arn{\'e} and Hochreiter, Sepp",
  title={{Large-scale comparison of machine learning methods for drug target prediction on ChEMBL}},
  journal="Chem. Sci.",

Electronic Supplementary Information
Technical Appendix and Partial Reanalysis


Benchmark Dataset



Static Features (MD5: b8d29262e7d1069052fdf2078e44e6d8)
Semisparse Features (MD5: 0458b111e5bf47b11d5fe3edb44fcd7f)
ECFP6 Features (MD5: f9bac94a3738f8e3c20ad4248832a767)
DFS8 Features (MD5: c26a01d4dbd9db7477f41b87ebfb3361)
Toxicophore Features (MD5: f293b77f62ab6a2dc910cd570b2b4c46)

ECFP4 Features (MD5: d09fca4b2a05a4aeed64375d49124f04, used for clustering)


Sample Map: SampleIndex to ChEMBLId (MD5: df7e773de4ce0272d8ed8207c4ef6a6f, index starting at 0)
Folds (MD5: 7187541a8706bdfbc32d53c6f936266c, columns: SampleIndex, Fold | Folds: 0, 1, 2)

SQL Query Result (MD5: 77789163dd0e75a0fb66c3b703bc0570, Extraction of Data from ChEMBL database with SQL query)
Application of Protocol for Activity Call (MD5: 7b2becf6f52d9f9798631e6529378fc1, columns: Activity, SampleIndex, Target | double entries may exist)

Meaning of Activity:
Activity 0: garbage measurement
Activity 1: inactive
Activity 2: indeterminate
Activity 3: active
Activity 10: weak garbage measurement
Activity 11: weakly inactive
Activity 13: weakely active

Used Assays (MD5: e5ee2c1c729f1af7190e1d220241a780)

License of our codes: LICENSE

Preprocessed data:

Preprocessed data in C/C++ format and Python format is available here.

C/C++ and Python Pipeline:

The individual steps are given in More details on these steps are described below.

Code to produce Benchmark Dataset:

available at GitHub
and at


created with jCompoundMapper (source code [r55]) with 2 files exchanged:

Static (dense) Features:

Semisparse features

  1. MACCS:
  2. PubChem:
  3. SHED, CATS2D:
  4. RDKit Topological Fingerprints (daylight):

Training Labels

Clustering ChEMBL

Considered Targets (=Assays)

Code to reproduce results:

available at GitHub

C/C++ Pipeline

Python Pipeline

Data analysis

Data analysis scripts are available here.

Contact: Andreas Mayr (
Last Update: July, 5th, 2018