Basemodel: GPT-Neo

Configs: Vocab size: 10,000 Hidden size: 512 Max position embeddings: 512 Number of layers: 2 Number of heads: 4 Window size: 256 Intermediate-size: 1024

Results:

  • Task: glue Score: 59.41 Confidence Interval: [59.0, 59.82]
  • Task: blimp Score: 56.91 Confidence Interval: [55.93, 57.8]
Downloads last month
54
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train AISE-TUDelft/Custom-Activations-GPT-ReLU

Collection including AISE-TUDelft/Custom-Activations-GPT-ReLU