Ensemble PyTorch

Ensemble PyTorch是一个统一的集成框架，用于pytorch_，可以轻松提高深度学习模型的性能和鲁棒性。Ensemble-PyTorch是pytorch生态系统<https://pytorch.org/ecosystem/>__的一部分，这要求项目得到良好的维护。

文档 <https://ensemble-pytorch.readthedocs.io/>__
实验 <https://ensemble-pytorch.readthedocs.io/en/stable/experiment.html>__

安装

.. code:: bash

pip install torchensemble

示例

.. code:: python

from torchensemble import VotingClassifier  # 投票是一种经典的集成策略

# 加载数据
train_loader = DataLoader(...)
test_loader = DataLoader(...)

# 定义集成
ensemble = VotingClassifier(
    estimator=base_estimator,               # estimator是你的pytorch模型
    n_estimators=10,                        # 基础估计器的数量
)

# 设置优化器
ensemble.set_optimizer(
    "Adam",                                 # 参数优化器的类型
    lr=learning_rate,                       # 参数优化器的学习率
    weight_decay=weight_decay,              # 参数优化器的权重衰减
)

# 设置学习率调度器
ensemble.set_scheduler(
    "CosineAnnealingLR",                    # 学习率调度器的类型
    T_max=epochs,                           # 调度器的其他参数
)

# 训练集成
ensemble.fit(
    train_loader,
    epochs=epochs,                          # 训练轮数
)

# 评估集成
acc = ensemble.evaluate(test_loader)         # 测试准确率

支持的集成方法

+------------------------------+------------+---------------------------+-----------------------------+ | 集成名称 | 类型 | 源代码 | 问题 | +==============================+============+===========================+=============================+ | 融合 | 混合 | fusion.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 投票 [1]_ | 并行 | voting.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 神经森林 | 并行 | voting.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 装袋 [2]_ | 并行 | bagging.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 梯度提升 [3]_ | 顺序 | gradient_boosting.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 快照集成 [4]_ | 顺序 | snapshot_ensemble.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 对抗训练 [5]_ | 并行 | adversarial_training.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 快速几何集成 [6]_ | 顺序 | fast_geometric.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+ | 软梯度提升 [7]_ | 并行 | soft_gradient_boosting.py | 分类 / 回归 | +------------------------------+------------+---------------------------+-----------------------------+

依赖

scikit-learn>=0.23.0
torch>=1.4.0
torchvision>=0.2.2

参考文献

.. [1] Zhou, Zhi-Hua. Ensemble Methods: Foundations and Algorithms. CRC press, 2012.

.. [2] Breiman, Leo. Bagging Predictors. Machine Learning (1996): 123-140.

.. [3] Friedman, Jerome H. Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics (2001): 1189-1232.

.. [4] Huang, Gao, et al. Snapshot Ensembles: Train 1, Get M For Free. ICLR, 2017.

.. [5] Lakshminarayanan, Balaji, et al. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. NIPS, 2017.

.. [6] Garipov, Timur, et al. Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs. NeurIPS, 2018.

.. [7] Feng, Ji, et al. Soft Gradient Boosting Machine. ArXiv, 2020.

.. _pytorch: https://pytorch.org/

.. _pypi: https://pypi.org/project/torchensemble/

感谢所有贡献者

|contributors|

.. |contributors| image:: https://contributors-img.web.app/image?repo=TorchEnsemble-Community/Ensemble-Pytorch .. _contributors: https://github.com/TorchEnsemble-Community/Ensemble-Pytorch/graphs/contributors