Overview
Scarlet-Seraph-12B is made by merging
Scarlet-Ink-12B,
Abyssal-Seraph-12B, and
Scarlet-Eclipse-12B
using a custom method.
Show YAML Config
models:
- model: Vortex5/Scarlet-Ink-12B
parameters:
weight:
- filter: self_attn
value: 0.1
- filter: mlp
value: 0.33
- value: 0.33
- model: Vortex5/Abyssal-Seraph-12B
parameters:
weight:
- filter: self_attn
value: 0.8
- filter: mlp
value: 0.4
- value: 0.33
- model: Vortex5/Scarlet-Eclipse-12B
parameters:
weight:
- filter: self_attn
value: 0.15
- filter: mlp
value: 0.45
- value: 0.33
merge_method: amsf
parameters:
normalize: true
dtype: bfloat16
tokenizer:
source: Vortex5/Abyssal-Seraph-12B
