From d8e5631567f8f007735b58e5297554a338c69e8f Mon Sep 17 00:00:00 2001 From: Michael Panchenko Date: Sun, 5 May 2024 22:26:49 +0200 Subject: [PATCH] Extended changelog, slightly improved structure --- CHANGELOG.md | 90 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 55 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33622ce0f..d74e9900b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,31 +3,48 @@ ## Release 1.1.0 ### Api Extensions -- Batch received two new methods: `to_dict` and `to_list_of_dicts`. #1063 -- `Collector`s can now be closed, and their reset is more granular. #1063 -- Trainers can control whether collectors should be reset prior to training. #1063 -- Convenience constructor for `CollectStats` called `with_autogenerated_stats`. #1063 -- `SamplingConfig` supports `batch_size=None`. #1077 -- Batch received new methods: `to_numpy_` and `to_torch_`. #1098, #1117 -- `to_dict` in Batch supports also non-recursive conversion. #1098 -- Batch `__eq__` implemented, semantic equality check of batches is now possible. #1098 -- `Batch.keys()` deprecated in favor of `Batch.get_keys()` (needed to make iteration consistent with naming) #1105. -- `Experiment` and `ExperimentConfig` now have a `name`, that can however be overridden when `Experiment.run()` is called. #1074 -- When building an `Experiment` from an `ExperimentConfig`, the user has the option to add info about seeds to the name. #1074 -- New method in `ExperimentConfig` called `build_default_seeded_experiments`. #1074 -- `SamplingConfig` has an explicit training seed, `test_seed` is inferred. #1074 -- New `evaluation` package for repeating the same experiment with multiple seeds and aggregating the results (important extension!). -Launchers for parallelization currently in alpha state. #1074 +- `data`: + - `Batch`: + - Add methods `to_dict` and `to_list_of_dicts`. #1063 #1098 + - Add methods `to_numpy_` and `to_torch_`. #1098, #1117 + - Add `__eq__` (semantic equality check). #1098 + - `keys()` deprecated in favor of `get_keys()` (needed to make iteration consistent with naming) #1105. + - `data.collector`: + - `Collector`: + - Introduced `BaseCollector` as a base class for all collectors. #1123 + - Add method `close` #1063 + - Method `reset` is now more granular (new flags controlling behavior). #1063 + - `CollectStats`: Add convenience constructor `with_autogenerated_stats`. #1063 +- `trainer`: + - Trainers can now control whether collectors should be reset prior to training. #1063 +- policy: + - introduced attribute `in_training_step` that is controlled by the trainer. #1123 + - policy automatically set to `eval` mode when collecting and to `train` mode when updating. #1123 +- `highlevel`: + - `SamplingConfig`: + - Add support for `batch_size=None`. #1077 + - Add `training_seed` for explicit seeding of training and test environments, the `test_seed` is inferred from `training_seed`. #1074 + - `highlevel.experiment`: + - `Experiment` now has a `name` attribute, which can be set using `ExperimentBuilder.with_name` and + which determines the default run name and therefore the persistence subdirectory. + It can still be overridden in `Experiment.run()`, the new parameter name being `run_name` rather than + `experiment_name` (although the latter will still be interpreted correctly). #1074 #1131 + - Add class `ExperimentCollection` for the convenient execution of multiple experiment runs #1131 + - `ExperimentBuilder`: + - Add method `build_seeded_collection` for the sound creation of multiple + experiments with varying random seeds #1131 + - Add method `copy` to facilitate the creation of multiple experiments from a single builder #1131 +- `evaluation`: New package for repeating the same experiment with multiple seeds and aggregating the results. #1074 + - The module `evaluation.launchers` for parallelization is currently in alpha state. - Loggers can now restore the logged data into python by using the new `restore_logged_data` method. #1074 -- `continuous.Critic`: - - Add flag `apply_preprocess_net_to_obs_only` to allow the - preprocessing network to be applied to the observations only (without - the actions concatenated), which is essential for the case where we want - to reuse the actor's preprocessing network #1128 -- Base class for collectors: `BaseCollector` #1122 -- Collectors can now explicitly specify whether to use the policy in training or evaluation mode. #1122 -- New util context managers `in_eval_mode` and `in_train_mode` for torch modules. #1122 -- `reset` of `Collectors` now returns `obs` and `info`. #1122 +- `utils`: + - `net.continuous.Critic`: + - Add flag `apply_preprocess_net_to_obs_only` to allow the + preprocessing network to be applied to the observations only (without + the actions concatenated), which is essential for the case where we want + to reuse the actor's preprocessing network #1128 + - `torch_utils` (new module) + - Added contextmanagers `in` ### Fixes - `CriticFactoryReuseActor`: Enable the Critic flag `apply_preprocess_net_to_obs_only` for continuous critics, @@ -52,25 +69,28 @@ instead of just `nn.Module`. #1032 - tests and examples are covered by `mypy`. #1077 - `NetBase` is more used, stricter typing by making it generic. #1077 - Use explicit multiprocessing context for creating `Pipe` in `subproc.py`. #1102 -- Removed all `if __name__ == "__main__":` blocks from tests. #1122 -- Improved typing issues in tests with buffer and collector. #1122 ### Breaking Changes - -- Removed `.data` attribute from `Collector` and its child classes. #1063 -- Collectors no longer reset the environment on initialization. Instead, the user might have to call `reset` -expicitly or pass `reset_before_collect=True` . #1063 +- `data`: + - `Collector`: + - Removed `.data` attribute. #1063 + - Collectors no longer reset the environment on initialization. + Instead, the user might have to call `reset` expicitly or pass `reset_before_collect=True` . #1063 + - Removed `no_grad` argument from `collect` method (was unused in tianshou). #1123 + - `Batch`: + - Fixed `iter(Batch(...)` which now behaves the same way as `Batch(...).__iter__()`. + Can be considered a bugfix. #1063 + - The methods `to_numpy` and `to_torch` in are not in-place anymore + (use `to_numpy_` or `to_torch_` instead). #1098, #1117 +- Logging: + - `BaseLogger.prepare_dict_for_logging` is now abstract. #1074 + - Removed deprecated and unused `BasicLogger` (only affects users who subclassed it). #1074 - VectorEnvs now return an array of info-dicts on reset instead of a list. #1063 -- Fixed `iter(Batch(...)` which now behaves the same way as `Batch(...).__iter__()`. Can be considered a bugfix. #1063 - Changed interface of `dist_fn` in `PGPolicy` and all subclasses to take a single argument in both continuous and discrete cases. #1032 - `utils.net.common.Recurrent` now receives and returns a `RecurrentStateBatch` instead of a dict. #1077 -- The methods `to_numpy` and `to_torch` in `Batch` is not in-place anymore (use `to_numpy_` or `to_torch_` instead). #1098, #1117 - `AtariEnvFactory` constructor (in examples, so not really breaking) now requires explicit train and test seeds. #1074 - `EnvFactoryRegistered` now requires an explicit `test_seed` in the constructor. #1074 -- `BaseLogger.prepare_dict_for_logging` is now abstract. #1074 -- Removed deprecated and unused `BasicLogger` (only affects users who subclassed it). #1074 -- Removed deprecations of `0.5.1` (will likely not affect anyone) and the unused `warnings` module. #1122 ### Tests