.. _pytorch_mode_half:

=====================
PyTorch FP16
=====================

PyTorch只在 :ref:`cuda` 中支持FP16，并且在新型的CPU或TPU中支持BF16 ( `PyTorch twitter 4/11 <https://twitter.com/PyTorch/status/1450502321838960641>`_ )

参考
======

- `FP16 in Pytorch <https://medium.com/@dwightfoster03/fp16-in-pytorch-a042e9967f7e>`_
- `Performance and Scalability: How To Fit a Bigger Model and Train It Faster <https://huggingface.co/docs/transformers/v4.15.0/performance>`_