1.6 KiB
1.6 KiB
Custom Layers and Utilities
This page lists all the custom layers used by the library, as well as the utility functions and classes it provides for modeling.
Most of those are only useful if you are studying the code of the models in the library.
Layers
autodoc GradientCheckpointingLayer
Attention Functions
autodoc AttentionInterface - register
Attention Mask Functions
autodoc AttentionMaskInterface - register
Rotary Position Embedding Functions
autodoc dynamic_rope_update
Pytorch custom modules
autodoc pytorch_utils.Conv1D
PyTorch Helper Functions
autodoc pytorch_utils.apply_chunking_to_forward
autodoc pytorch_utils.find_pruneable_heads_and_indices
autodoc pytorch_utils.prune_layer
autodoc pytorch_utils.prune_conv1d_layer
autodoc pytorch_utils.prune_linear_layer