LoRA-DR-suite is a family of models for the identification of disordered regions (DR) in proteins, built upon state-of-the-art Protein Language Models (PLMs) trained on protein sequences only. They leverage Low-Rank Adaptation (LoRA) fine-tuning for binary classification of intrinsic and soft disorder.
Intrinsically-disordered residues are experimentally detected through circular dichroism and X-ray cristallography, while soft disorder is characterized by high B-factor, or intermittently
missing residues across different X-ray crystal structures of the same sequence.
Models for intrinsic disorder are trained on DisProt 7.0 data only (DisProt7 suffix) or on additional data from the first and second edition of the Critical Assesment of Intrinsic Disorder (CAID), indicated with the ID suffix.
Models for soft disorder classification are trained instead on the SoftDis dataset, derived from an extensive analysis of clusters of alternative structures for the same protein
sequence in the Protein Data Bank (PDB). For each position in the represantitive sequence of each cluster, it provides the frequency of closely-related homologs for which the corresponding residue is higly flexible or missing.
## Repository content
This repository provides code for hyperparameters optimization (HPO) and training data for the LoRA-DR-suite models. In addition, jupyter notebooks show how to reproduce performances for intrinsic and soft disorder prediction tests, and how to load and process data from SoftDis dataset.
In order to run the code, clone the repository and install the required packages in a dedicated `conda` environment with `pip install -r requirements.txt`. Referenced PyTorch versions requires CUDA 12.1.
HPO and training main script is contained in `hpo_optuna.py`. Required parameters can be inspected with the `--help` option. Example HPO configurations for the different pre-trained models and tasks are saved in `config` folder.
## Model checkpoints
We provide different model checkpoints for LoRA-DR-suite models,, based on training data and pre-trained PLM, that can be downloaded and accessed on HuggingFace Hub.
| Checkpoint name | Training dataset | Pre-trained checkpoint |
The models are intended to be used for classification of different disorder types.
Models for intrinsic disorder trained on DisProt 7.0 were evaluated on CAID1 and CAID2 challenge, but we suggest to use "ID" models for classification of new sequences, as they show better generalization.
In addition to its relation to flexibility and assembly pathways, soft disorder can be used to infer confidence score for structure prediciton tools, as we found high negative Spearman correlation between soft disorder probabilities and pLDDT from AlphaFold2 predicitons.
### Model usage
All models can be loaded as PyTorch Modules, together with their associated tokenizer, with the following code:
