Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

How can I force SMAC to try the default configurations? #1140

Open
Strawberry9583 opened this issue Aug 20, 2024 · 2 comments
Open

How can I force SMAC to try the default configurations? #1140

Strawberry9583 opened this issue Aug 20, 2024 · 2 comments

Comments

@Strawberry9583
Copy link

Description

I have an unexpected case when I am using SMAC2.1.0.

My tuned version of the solver is even worse than the default configured versions, even though I set the default configuration when running SMAC. We tried to get improved performance of geode and or-tools but failed. I also checked and tried the configuration during SMAC running; the default configuration hasn't been tried at all.

Are there any methods that I can ask SMAC to try the default configuration?

Steps/Code to Reproduce

SMAC2.1.0

Expected Results

SMAC can run the default configuration I have set in the configuration space at first. Then SMAC tried other configurations.

Actual Results

SMAC never tried the default configuration in the whole optimization period.

Versions

@benjamc
Copy link
Collaborator

benjamc commented Aug 27, 2024

Could you please add a minimal working example how you exactly have setup SMAC to use the default configuration?
In general, you can pass initial_design=DefaultInitialDesign() to the facade (from smac.initial_design import DefaultInitialDesign).

@benjamc
Copy link
Collaborator

benjamc commented Sep 25, 2024

You can do something like this:

...
configurations = [your_default_configuration]
smac = HyperparameterOptimizationFacade(
        scenario,
        task.evaluate,
        overwrite=True,

        # Modify the initial design to use our custom initial design
        initial_design=HyperparameterOptimizationFacade.get_initial_design(
            scenario, 
            n_configs=0,  # Do not use the default initial design
            additional_configs=configurations  # Use the configurations previously evaluated as initial design
                                               # This only passes the configurations but not the cost!
        )
    )

(from the warmstart example )

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants