Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Cleanup #1422

Open
wants to merge 19 commits into
base: develop
Choose a base branch
from
+401 −642
Open

Cleanup #1422

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
49ca436
DCE, rename variable
MasterSkepticista Mar 5, 2025
831e724
Move to module-level loggers
MasterSkepticista Mar 5, 2025
6283c48
Plan.logger to logger
MasterSkepticista Mar 5, 2025
636f8e3
Fixup
MasterSkepticista Mar 5, 2025
41e4793
Also for workspaces
MasterSkepticista Mar 5, 2025
84a4323
Fix again for logger
MasterSkepticista Mar 5, 2025
818f225
Remove run_simulation tests
MasterSkepticista Mar 5, 2025
2372d1e
Header creation is stateless and common across both
MasterSkepticista Mar 5, 2025
c878271
Remove TASK_REGISTRY code; deprecated with interactive
MasterSkepticista Mar 5, 2025
dcf6562
More use_delta_updates changes
MasterSkepticista Mar 5, 2025
9855164
lets see if this revert works
MasterSkepticista Mar 5, 2025
322fa3e
Remove openfl install in notebook
MasterSkepticista Mar 5, 2025
ea67b01
revert few flags
MasterSkepticista Mar 5, 2025
8bc086e
Merge branch 'develop' into karansh1/dce
MasterSkepticista Mar 5, 2025
50ebdef
please please please this should work
MasterSkepticista Mar 5, 2025
bcc4f04
Migrate stateless function to common
MasterSkepticista Mar 6, 2025
268737b
Cleanup outdated docstrings
MasterSkepticista Mar 6, 2025
dbbf755
Remove Interactive API RPCs
MasterSkepticista Mar 6, 2025
512e018
Remove set devices call; not used anywhere
MasterSkepticista Mar 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/about/features_index/secure_aggregation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The following plan shows secure aggregation being enabled on `keras/mnist <https
collaborator:
settings:
db_store_rounds: 1
delta_updates: false
use_delta_updates: false
opt_treatment: RESET
template: openfl.component.Collaborator
compression_pipeline:
Expand Down
2 changes: 1 addition & 1 deletion docs/about/features_index/taskrunner.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Configurable Settings
`openfl.component.Collaborator <https://github.com/intel/openfl/blob/develop/openfl/component/collaborator/collaborator.py>`_
Defines the settings for the collaborator which is the data owner in the experiment. The settings for collaborator include:

- :code:`delta_updates`: (boolean) Determines whether the difference in model weights between the current and previous round will be sent (True), or if whole checkpoints will be sent (False). Setting to delta_updates to True leads to higher sparsity in model weights sent across, which may improve compression ratios.
- :code:`use_delta_updates`: (boolean) Determines whether the difference in model weights between the current and previous round will be sent (True), or if whole checkpoints will be sent (False). Setting to use_delta_updates to True leads to higher sparsity in model weights sent across, which may improve compression ratios.
- :code:`opt_treatment`: (str) Defines the optimizer state treatment policy. Valid options are : 'RESET' - reinitialize optimizer for every round (default), 'CONTINUE_LOCAL' - keep local optimizer state for every round, 'CONTINUE_GLOBAL' - aggregate optimizer state for every round.


Expand Down
1 change: 0 additions & 1 deletion openfl-tutorials/experimental/workflow/301_MNIST_Watermarking.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@
"metadata": {},
"outputs": [],
"source": [
"!pip install git+https://github.com/securefederatedai/openfl.git\n",
"!pip install -r workflow_interface_requirements.txt\n",
"!pip install torch\n",
"!pip install torchvision\n",
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/gandlf_seg_test/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/keras/2dunet/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : true
use_delta_updates : true
opt_treatment : RESET

data_loader :
Expand Down
4 changes: 3 additions & 1 deletion openfl-workspace/keras/2dunet/src/taskrunner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@

from openfl.federated import KerasTaskRunner

import logging
logger = logging.getLogger(__name__)

class Keras2DUNet(KerasTaskRunner):
"""Initialize.
Expand All @@ -27,7 +29,7 @@ def __init__(self, **kwargs):
super().__init__(**kwargs)

self.model = self.build_model(self.data_loader.get_feature_shape(), use_upsampling=True, **kwargs)
self.model.summary(print_fn=self.logger.info, line_length=120)
self.model.summary(print_fn=logger.info, line_length=120)
self.initialize_tensorkeys_for_functions()


Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/keras/jax/mnist/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
9 changes: 6 additions & 3 deletions openfl-workspace/keras/jax/mnist/src/taskrunner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
from .model import CNNModel
from openfl.federated import KerasTaskRunner

import logging
logger = logging.getLogger(__name__)

class JAXCNN(KerasTaskRunner):
"""A basic convolutional neural network model."""

Expand All @@ -24,10 +27,10 @@ def __init__(self, **kwargs):

self.initialize_tensorkeys_for_functions()

self.model.summary(print_fn=self.logger.info)
self.model.summary(print_fn=logger.info)

self.logger.info(f'Train Set Size : {self.get_train_data_size()}')
self.logger.info(f'Valid Set Size : {self.get_valid_data_size()}')
logger.info(f'Train Set Size : {self.get_train_data_size()}')
logger.info(f'Valid Set Size : {self.get_valid_data_size()}')

def build_model(self,
input_shape,
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/keras/mnist/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
8 changes: 5 additions & 3 deletions openfl-workspace/keras/mnist/src/taskrunner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@

from openfl.federated import KerasTaskRunner

import logging
logger = logging.getLogger(__name__)

class KerasCNN(KerasTaskRunner):
"""A basic convolutional neural network model."""
Expand All @@ -27,10 +29,10 @@ def __init__(self, **kwargs):

self.initialize_tensorkeys_for_functions()

self.model.summary(print_fn=self.logger.info)
self.model.summary(print_fn=logger.info)

self.logger.info(f'Train Set Size : {self.get_train_data_size()}')
self.logger.info(f'Valid Set Size : {self.get_valid_data_size()}')
logger.info(f'Train Set Size : {self.get_train_data_size()}')
logger.info(f'Valid Set Size : {self.get_valid_data_size()}')

def build_model(self,
input_shape,
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/keras/tensorflow/mnist/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
9 changes: 6 additions & 3 deletions openfl-workspace/keras/tensorflow/mnist/src/taskrunner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
from .model import CNNModel
from openfl.federated import KerasTaskRunner

import logging
logger = logging.getLogger(__name__)

class CNNTaskruner(KerasTaskRunner):
"""A basic convolutional neural network model."""

Expand Down Expand Up @@ -35,10 +38,10 @@ def __init__(self, **kwargs):

self.initialize_tensorkeys_for_functions()

self.model.summary(print_fn=self.logger.info)
self.model.summary(print_fn=logger.info)

self.logger.info(f'Train Set Size : {self.get_train_data_size()}')
self.logger.info(f'Valid Set Size : {self.get_valid_data_size()}')
logger.info(f'Train Set Size : {self.get_train_data_size()}')
logger.info(f'Valid Set Size : {self.get_valid_data_size()}')

def build_model(self,
input_shape,
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/keras/torch/mnist/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
10 changes: 7 additions & 3 deletions openfl-workspace/keras/torch/mnist/src/taskrunner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@

from .model import CNNModel
from openfl.federated import KerasTaskRunner

import logging
logger = logging.getLogger(__name__)

class KerasCNN(KerasTaskRunner):
"""A basic convolutional neural network model."""

Expand All @@ -23,10 +27,10 @@ def __init__(self, **kwargs):

self.initialize_tensorkeys_for_functions()

self.model.summary(print_fn=self.logger.info)
self.model.summary(print_fn=logger.info)

self.logger.info(f'Train Set Size : {self.get_train_data_size()}')
self.logger.info(f'Valid Set Size : {self.get_valid_data_size()}')
logger.info(f'Train Set Size : {self.get_train_data_size()}')
logger.info(f'Valid Set Size : {self.get_valid_data_size()}')

def build_model(self,
input_shape,
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/histology/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/histology_fedcurv/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/mnist/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ assigner :
collaborator:
settings:
db_store_rounds: 1
delta_updates: false
use_delta_updates: false
opt_treatment: RESET
template: openfl.component.collaborator.Collaborator
compression_pipeline:
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/mnist_eden_compression/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/mnist_fed_eval/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/mnist_straggler_check/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/torch/template/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.collaborator.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
2 changes: 1 addition & 1 deletion openfl-workspace/workspace/plan/defaults/collaborator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
template : openfl.component.Collaborator
settings :
opt_treatment : 'CONTINUE_LOCAL'
delta_updates : True
use_delta_updates : True
db_store_rounds : 1
2 changes: 1 addition & 1 deletion openfl-workspace/xgb_higgs/plan/plan.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ collaborator :
defaults : plan/defaults/collaborator.yaml
template : openfl.component.collaborator.Collaborator
settings :
delta_updates : false
use_delta_updates : false
opt_treatment : RESET

data_loader :
Expand Down
Loading
Loading