Skip to content

Commit 4bfb39c

Browse files
author
Github Actions
committed
ArlindKadra: Updating implementation of the reg cocktails so that it is compatible with fixed search space updates
1 parent 99a1ebe commit 4bfb39c

19 files changed

+807
-497
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt

+66-72
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7f02e2e973d0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7efeb6ade400> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -87,7 +87,7 @@ the search. Currently, there are two changes that can be made to the space:-
8787
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
8888
trainer:StandardTrainer:weighted_loss, Value: True
8989
trainer:__choice__, Value: 'StandardTrainer'
90-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002084016799926758, budget=0), TrajEntry(train_perf=0.14035087719298245, incumbent_id=1, incumbent=Configuration:
90+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001983165740966797, budget=0), TrajEntry(train_perf=0.15204678362573099, incumbent_id=1, incumbent=Configuration:
9191
data_loader:batch_size, Value: 32
9292
encoder:__choice__, Value: 'OneHotEncoder'
9393
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -128,21 +128,57 @@ the search. Currently, there are two changes that can be made to the space:-
128128
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
129129
trainer:StandardTrainer:weighted_loss, Value: True
130130
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=1, ta_time_used=5.20469331741333, wallclock_time=6.656493186950684, budget=5.555555555555555)]
132-
{'accuracy': 0.8786127167630058}
133-
| | Preprocessing | Estimator | Weight |
134-
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------|---------:|
135-
| 0 | None | CatBoostClassifier | 0.2 |
136-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
137-
| 2 | None | ExtraTreesClassifier | 0.14 |
138-
| 3 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 |
139-
| 4 | SimpleImputer,OneHotEncoder,NoScaler,PolynomialFeatures | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
140-
| 5 | None | LGBMClassifier | 0.08 |
141-
| 6 | None | RFClassifier | 0.06 |
142-
| 7 | None | KNNClassifier | 0.06 |
143-
| 8 | None | SVC | 0.02 |
144-
| 9 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
145-
<smac.runhistory.runhistory.RunHistory object at 0x7f02e0a1c6a0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
131+
, ta_runs=1, ta_time_used=5.583842515945435, wallclock_time=7.051768779754639, budget=5.555555555555555), TrajEntry(train_perf=0.1871345029239766, incumbent_id=2, incumbent=Configuration:
132+
data_loader:batch_size, Value: 409
133+
encoder:__choice__, Value: 'OneHotEncoder'
134+
feature_preprocessor:TruncatedSVD:target_dim, Value: 4
135+
feature_preprocessor:__choice__, Value: 'TruncatedSVD'
136+
imputer:categorical_strategy, Value: 'most_frequent'
137+
imputer:numerical_strategy, Value: 'constant_zero'
138+
lr_scheduler:CosineAnnealingLR:T_max, Value: 53
139+
lr_scheduler:__choice__, Value: 'CosineAnnealingLR'
140+
network_backbone:MLPBackbone:activation, Value: 'relu'
141+
network_backbone:MLPBackbone:dropout_1, Value: 0.7173131362280667
142+
network_backbone:MLPBackbone:dropout_2, Value: 0.32527643996037753
143+
network_backbone:MLPBackbone:num_groups, Value: 2
144+
network_backbone:MLPBackbone:num_units_1, Value: 165
145+
network_backbone:MLPBackbone:num_units_2, Value: 244
146+
network_backbone:MLPBackbone:use_dropout, Value: True
147+
network_backbone:__choice__, Value: 'MLPBackbone'
148+
network_embedding:__choice__, Value: 'NoEmbedding'
149+
network_head:__choice__, Value: 'fully_connected'
150+
network_head:fully_connected:activation, Value: 'tanh'
151+
network_head:fully_connected:num_layers, Value: 3
152+
network_head:fully_connected:units_layer_1, Value: 256
153+
network_head:fully_connected:units_layer_2, Value: 408
154+
network_init:OrthogonalInit:bias_strategy, Value: 'Normal'
155+
network_init:__choice__, Value: 'OrthogonalInit'
156+
optimizer:RMSpropOptimizer:alpha, Value: 0.44965916725624266
157+
optimizer:RMSpropOptimizer:lr, Value: 0.0015378712542410064
158+
optimizer:RMSpropOptimizer:momentum, Value: 0.7354696550868378
159+
optimizer:RMSpropOptimizer:use_weight_decay, Value: True
160+
optimizer:RMSpropOptimizer:weight_decay, Value: 3.025630486858573e-06
161+
optimizer:__choice__, Value: 'RMSpropOptimizer'
162+
scaler:__choice__, Value: 'StandardScaler'
163+
trainer:RowCutOutTrainer:cutout_prob, Value: 0.589936979878005
164+
trainer:RowCutOutTrainer:patch_ratio, Value: 0.40724571391547737
165+
trainer:RowCutOutTrainer:use_lookahead_optimizer, Value: False
166+
trainer:RowCutOutTrainer:use_snapshot_ensemble, Value: False
167+
trainer:RowCutOutTrainer:use_stochastic_weight_averaging, Value: True
168+
trainer:RowCutOutTrainer:weighted_loss, Value: True
169+
trainer:__choice__, Value: 'RowCutOutTrainer'
170+
, ta_runs=13, ta_time_used=87.67640423774719, wallclock_time=120.08723640441895, budget=50.0)]
171+
{'accuracy': 0.861271676300578}
172+
| | Preprocessing | Estimator | Weight |
173+
|---:|:--------------------------------------------------------|:-------------------------------------------------------------|---------:|
174+
| 0 | None | RFClassifier | 0.28 |
175+
| 1 | None | ExtraTreesClassifier | 0.24 |
176+
| 2 | SimpleImputer,OneHotEncoder,NoScaler,PolynomialFeatures | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
177+
| 3 | None | CatBoostClassifier | 0.18 |
178+
| 4 | None | KNNClassifier | 0.06 |
179+
| 5 | SimpleImputer,OneHotEncoder,StandardScaler,TruncSVD | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
180+
| 6 | SimpleImputer,OneHotEncoder,StandardScaler,TruncSVD | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
181+
<smac.runhistory.runhistory.RunHistory object at 0x7efeb7182220> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
146182
data_loader:batch_size, Value: 32
147183
encoder:__choice__, Value: 'NoEncoder'
148184
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -181,7 +217,7 @@ the search. Currently, there are two changes that can be made to the space:-
181217
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
182218
trainer:StandardTrainer:weighted_loss, Value: True
183219
trainer:__choice__, Value: 'StandardTrainer'
184-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0016298294067382812, budget=0), TrajEntry(train_perf=0.20467836257309946, incumbent_id=1, incumbent=Configuration:
220+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0017466545104980469, budget=0), TrajEntry(train_perf=0.2222222222222222, incumbent_id=1, incumbent=Configuration:
185221
data_loader:batch_size, Value: 32
186222
encoder:__choice__, Value: 'NoEncoder'
187223
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -220,59 +256,17 @@ the search. Currently, there are two changes that can be made to the space:-
220256
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
221257
trainer:StandardTrainer:weighted_loss, Value: True
222258
trainer:__choice__, Value: 'StandardTrainer'
223-
, ta_runs=1, ta_time_used=4.402615785598755, wallclock_time=5.924044609069824, budget=5.555555555555555), TrajEntry(train_perf=0.18128654970760238, incumbent_id=2, incumbent=Configuration:
224-
data_loader:batch_size, Value: 215
225-
encoder:__choice__, Value: 'NoEncoder'
226-
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
227-
imputer:categorical_strategy, Value: 'constant_!missing!'
228-
imputer:numerical_strategy, Value: 'most_frequent'
229-
lr_scheduler:CosineAnnealingLR:T_max, Value: 51
230-
lr_scheduler:__choice__, Value: 'CosineAnnealingLR'
231-
network_backbone:ShapedResNetBackbone:activation, Value: 'tanh'
232-
network_backbone:ShapedResNetBackbone:blocks_per_group, Value: 2
233-
network_backbone:ShapedResNetBackbone:max_dropout, Value: 0.37376142861116285
234-
network_backbone:ShapedResNetBackbone:max_units, Value: 106
235-
network_backbone:ShapedResNetBackbone:multi_branch_choice, Value: 'None'
236-
network_backbone:ShapedResNetBackbone:num_groups, Value: 1
237-
network_backbone:ShapedResNetBackbone:output_dim, Value: 979
238-
network_backbone:ShapedResNetBackbone:resnet_shape, Value: 'stairs'
239-
network_backbone:ShapedResNetBackbone:use_batch_norm, Value: False
240-
network_backbone:ShapedResNetBackbone:use_dropout, Value: True
241-
network_backbone:ShapedResNetBackbone:use_skip_connection, Value: True
242-
network_backbone:__choice__, Value: 'ShapedResNetBackbone'
243-
network_embedding:__choice__, Value: 'NoEmbedding'
244-
network_head:__choice__, Value: 'fully_connected'
245-
network_head:fully_connected:num_layers, Value: 1
246-
network_init:XavierInit:bias_strategy, Value: 'Normal'
247-
network_init:__choice__, Value: 'XavierInit'
248-
optimizer:RMSpropOptimizer:alpha, Value: 0.21475647146537818
249-
optimizer:RMSpropOptimizer:lr, Value: 0.06800579072253359
250-
optimizer:RMSpropOptimizer:momentum, Value: 0.19455509625221876
251-
optimizer:RMSpropOptimizer:use_weight_decay, Value: False
252-
optimizer:__choice__, Value: 'RMSpropOptimizer'
253-
scaler:Normalizer:norm, Value: 'max'
254-
scaler:__choice__, Value: 'Normalizer'
255-
trainer:RowCutMixTrainer:Lookahead:la_alpha, Value: 0.6084931276595258
256-
trainer:RowCutMixTrainer:Lookahead:la_steps, Value: 8
257-
trainer:RowCutMixTrainer:alpha, Value: 0.6686915138304311
258-
trainer:RowCutMixTrainer:se_lastk, Constant: 3
259-
trainer:RowCutMixTrainer:use_lookahead_optimizer, Value: True
260-
trainer:RowCutMixTrainer:use_snapshot_ensemble, Value: True
261-
trainer:RowCutMixTrainer:use_stochastic_weight_averaging, Value: True
262-
trainer:RowCutMixTrainer:weighted_loss, Value: False
263-
trainer:__choice__, Value: 'RowCutMixTrainer'
264-
, ta_runs=18, ta_time_used=125.97386693954468, wallclock_time=184.2776644229889, budget=50.0)]
265-
{'accuracy': 0.8728323699421965}
266-
| | Preprocessing | Estimator | Weight |
267-
|---:|:----------------------------------------------------------|:-------------------------------------------------------------------|---------:|
268-
| 0 | None | ExtraTreesClassifier | 0.22 |
269-
| 1 | None | CatBoostClassifier | 0.2 |
270-
| 2 | None | RFClassifier | 0.2 |
271-
| 3 | SimpleImputer,NoEncoder,NoScaler,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
272-
| 4 | None | KNNClassifier | 0.08 |
273-
| 5 | SimpleImputer,NoEncoder,Normalizer,Nystroem | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
274-
| 6 | SimpleImputer,NoEncoder,Normalizer,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
275-
| 7 | None | LGBMClassifier | 0.02 |
259+
, ta_runs=1, ta_time_used=4.803942680358887, wallclock_time=6.263315200805664, budget=5.555555555555555)]
260+
{'accuracy': 0.8670520231213873}
261+
| | Preprocessing | Estimator | Weight |
262+
|---:|:--------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
263+
| 0 | None | RFClassifier | 0.38 |
264+
| 1 | None | ExtraTreesClassifier | 0.22 |
265+
| 2 | None | CatBoostClassifier | 0.2 |
266+
| 3 | None | KNNClassifier | 0.1 |
267+
| 4 | SimpleImputer,NoEncoder,Normalizer,NoFeaturePreprocessing | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
268+
| 5 | None | SVC | 0.04 |
269+
| 6 | SimpleImputer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
276270
277271
278272
@@ -401,7 +395,7 @@ the search. Currently, there are two changes that can be made to the space:-
401395
402396
.. rst-class:: sphx-glr-timing
403397

404-
**Total running time of the script:** ( 11 minutes 41.996 seconds)
398+
**Total running time of the script:** ( 11 minutes 51.552 seconds)
405399

406400

407401
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

0 commit comments

Comments
 (0)