mohammadali.sadraei
/
SuperposPrompt_Thesis
Watch 1
Star 0
Fork 0
Code Issues 0 Pull Requests 0 Releases 0 Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1 Commit
1 Branch
Branch: main
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'main'
${ noResults }
SuperposPrompt_Thesis/09_Cluster/config5.yaml

config5.yaml 780B
Raw Permalink Blame History

123456789101112131415161718192021222324252627282930313233343536373839 
  1. default: &default

  2.   use_tqdm: true 

  3.   random_seed: 42

  4.   base_save_path: /home/msadraei/trained_final

  5.   model_name:  google/t5-base-lm-adapt

  6.   project_name_prefix: iclr_softmax_effect

  7.   experiment_name_suffix: null

  8.   train_batch_size: 24

  9.   valid_batch_size: 24

  10.   remove_dropout: true

  11.   learning_rate: 0.01

  12.   weight_decay: 0.01

  13.   num_epochs: 20

  14.   peft_params: null  # no mutation

  15.   hot_modules:

  16.   - sadcl

  17.   best_finder:

  18.     save: True

  19.     metric: valid_mean

  20.     higher_better: true

  21.   tasks:

  22.   - glue:qqp

  23.   - glue:qnli

  24.   - glue:mnli

  25.   - glue:sst2

  26. 

  27. run_configs:

  28. - <<: *default

  29.   peft_params:

  30.     kind: combine

  31.     n_tokens: 10

  32.     n_comb_tokens: 128

  33.     softmax: true

  34. # - <<: *default

  35. #   learning_rate: 0.3

  36. #   peft_params:

  37. #     kind: residual

  38. #     n_tokens: 10

  39. #     mlp_size: 128