winglian
/

omega-3b-8k

winglian commited on Sep 13, 2023

Commit

7f2fc92

•

1 Parent(s): c80c6e7

Update modeling_mixformer_sequential.py

Files changed (1) hide show

modeling_mixformer_sequential.py CHANGED Viewed

@@ -94,9 +94,6 @@ class RotaryEmbedding(nn.Module):
     ) -> None:
         super().__init__()
-        if scale_base is not None:
-            raise NotImplementedError
         # Generate and save the inverse frequency buffer (non-trainable)
         self.dim = dim
         self.base = base

     ) -> None:
         super().__init__()
         # Generate and save the inverse frequency buffer (non-trainable)
         self.dim = dim
         self.base = base