LAP/README.md

# LAP
An Attention-Based Module for Faithful Interpretation and Knowledge Injection in Convolutional Neural Networks

In this supplementary material, we have provided a pdf containing more details for some of the sections mentioned in the paper, and the code of our work.

# Data Preparation

## Metadata

First, download the metadata from [here](https://mega.nz/file/MqhXiLgC#NK1vd9ZLGU-3a182x-BXfvGCSuqHgq9hflI6tddh4Wc)

## RSNA

## CelebA

1. Download the dataset from [here](https://www.kaggle.com/jessicali9530/celeba-dataset)

2. Extract the data

    ```bash
    unzip -q -j celeba-dataset.zip '**/*.jpg' -d data/celeba
    ```

3. Extract `celeba.tsv` from [this](#metadata) metadata file to `dataset_metadata/celeba.tsv`

## Imagenet

1. Download the ImageNet ILSVRC2012 dataset from [here](https://www.image-net.org/challenges/LSVRC/index.php).

1. Extract `ILSVRC2012_img_train_t3.tar` to `data/imagenet/tars`.

1. Run the following command to extract the train images per class:

    ```bash
    python data_preparation/imagenet/extract_train.py -s data/imagenet/tars/ -n <num-threads>
    ```

1. Run the following command to extract the validation images per class (replace directory with the directory containing `ILSVRC2012_img_val.tar`, and extract `imagenet_val_maps.pklz` from [this](#metadata) metadata file):

    ```bash
    python data_preparation/imagenet/extract_val.py -s <directory> -m <imagenet_val_maps.pklz>
    ```

1. download the localizations bounding-boxes from [here](https://www.kaggle.com/c/imagenet-object-localization-challenge/data). Put `LOC_train_solution.csv` and `LOC_val_solution.csv` in `data/imagenet/extracted`.

# Model checkpoints

Download all the checkpoints from [here](https://mega.nz/file/16pWWA5L#b1SwLEv-YO5lL9wLVTcgykn2LmUBHm1exPFNB0JBoZo).

# Run

## Evaluate the checkpoints

### RSNA

#### Vanilla ResNet 18

##### Evaluate the model performance on the test set

```bash
python evaluate.py rsna.org_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.org_resnet.pth
```

#### Vanilla Inception V3

##### Evaluate the model performance on the test set

```bash
python evaluate.py rsna.org_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.org_inception.pth
```

#### WS ResNet 18

##### Evaluate the model performance on the test set

```bash
python evaluate.py rsna.ws_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_resnet.pth
```

##### Generate the LAP interpretations for positive samples

```bash
python interpret.py rsna.ws_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_resnet.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 1 --save-by-file-name --report-dir rsna/ws_resnet_interpretations
```

##### Generate the LAP interpretations for negative samples

```bash
python interpret.py rsna.ws_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_resnet.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 0 --save-by-file-name --report-dir rsna/ws_resnet_interpretations
```

#### WS Inception V3

##### Evaluate the model performance on the test set

```bash
python evaluate.py rsna.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_inception.pth
```

##### Generate the LAP interpretations for positive samples

```bash
python interpret.py rsna.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_inception.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 1 --save-by-file-name --report-dir rsna/ws_inception_interpretations
```

##### Generate the LAP interpretations for negative samples

```bash
python interpret.py rsna.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_inception.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 0 --save-by-file-name --report-dir rsna/ws_inception_interpretations
```

#### BB ResNet 18

##### Evaluate the model performance on the test set

```bash
python evaluate.py rsna.bb_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.bb_resnet.pth
```

##### Generate the LAP interpretations for positive samples

```bash
python interpret.py rsna.bb_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.bb_resnet.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 1 --save-by-file-name --report-dir rsna/bb_resnet_interpretations
```

##### Generate the LAP interpretations for negative samples

```bash
python interpret.py rsna.bb_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.bb_resnet.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 0 --save-by-file-name --report-dir rsna/bb_resnet_interpretations
```

#### BB Inception V3

##### Evaluate the model performance on the test set

```bash
python evaluate.py rsna.bb_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.bb_inception.pth
```

##### Generate the LAP interpretations for positive samples

```bash
python interpret.py rsna.bb_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.bb_inception.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 1 --save-by-file-name --report-dir rsna/bb_inception_interpretations
```

##### Generate the LAP interpretations for negative samples

```bash
python interpret.py rsna.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/rsna.ws_inception.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 0 --save-by-file-name --report-dir rsna/ws_inception_interpretations
```

### CelebA

#### Vanilla ResNet 18

##### Evaluate the model performance on the test set

```bash
python evaluate.py celeba.org_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.org_resnet.pth
```

#### Vanilla Inception V3

##### Evaluate the model performance on the test set

```bash
python evaluate.py celeba.org_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.org_inception.pth
```

#### WS ResNet 18

##### Evaluate the model performance on the test set

```bash
python evaluate.py celeba.ws_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.ws_resnet.pth
```

##### Generate the LAP interpretations for positive samples

```bash
python interpret.py celeba.ws_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.ws_resnet.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 1 --save-by-file-name --report-dir celeba/ws_resnet_interpretations
```

##### Generate the LAP interpretations for negative samples

```bash
python interpret.py celeba.ws_lap_resnet --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.ws_resnet.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 0 --save-by-file-name --report-dir celeba/ws_resnet_interpretations
```

#### WS Inception V3

##### Evaluate the model performance on the test set

```bash
python evaluate.py celeba.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.ws_inception.pth
```

##### Generate the LAP interpretations for positive samples

```bash
python interpret.py celeba.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.ws_inception.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 1 --save-by-file-name --report-dir celeba/ws_inception_interpretations
```

##### Generate the LAP interpretations for negative samples

```bash
python interpret.py celeba.ws_lap_inception --device cuda:0 --samples-dir test --final-model-dir checkpoints/celeba.ws_inception.pth --skip-raw --interpretation-method Attention --global-threshold --cut-threshold 0 --mapped-labels-to-use 0 --save-by-file-name --report-dir celeba/ws_inception_interpretations
```

### ImageNet

#### WS ResNet 50 (Fine-tuned)

##### Evaluate the model performance on the validation set

```bash
python evaluate.py imagenet.lap_resnet50_ft --device cuda:0 --samples-dir val --final-model-dir checkpoints/imagenet.ws_resnet_ft.pth
```