Features

Enables differential‑privacy (DP) training of PyTorch models with little code change

Provides a `PrivacyEngine` API to wrap model, optimizer and data loader

Tracks privacy budget in real time during training

Supports advanced DP features such as Fast Gradient Clipping and Ghost Clipping

v1.6.0 Breaking risk 29d

Breaking changes

Notable features

Non-wrapping mode (`wrap_model=False`) for better interoperability with Transformers, Accelerate, and other libraries
Arithmetic operations support in `DPTensorFastGradientClipping` for custom loss compositions
Fully Sharded Data Parallel (FSDP) training support including tutorial and examples

Full changelog

New features

Add non-wrapping mode for better compatibility with Transformers, Accelerate, and libraries that expect the original module hierarchy (wrap_model=False) (#794)
Add arithmetic operations support to DPTensorFastGradientClipping, making it easier to integrate Opacus with custom loss compositions and external trainers (#805)

Add support for Fully Sharded Data Parallel (FSDP) training, including a tutorial and a new example (#761,#772,#781,#782)
Add support for mixed and low precision training (#764)
Add 1D tensor parallelism support for fast gradient clipping, together with toy and Llama examples; this support is currently beta (#776)

Bug fixes

Fix epsilon/noise accounting when using adaptive gradient clipping (#807, #779)
Fix fast gradient clipping when using ignore_index masking, so ignored tokens do not affect the reduced loss incorrectly (#808)
Replace empty-batch handling inside DPDataLoader with a structure-aware approach, fixing failures for custom batch structures under Poisson sampling (#806)
Treat IAccountant.mechanism correctly during state_dict handling (#778)

About

Stars

1,932

Forks

397

Languages