# cmix-neif-2.0: bf16 training state of the transformer This folder was made available on request, following a private message in the encode.su forum. It is not part of the entry cmix-neif-2.0 and is not needed to build or run it. | file | size | content | |---|---:|---| | `b3_e0400_model.tch` | 37,233,887 B | transformer weights at the end of stage b3 (400 epochs bf16) | | `b3_e0400_opt.pt` | 38,052,611 B | optimizer state of the same step | `b3_e0400_model.tch` is the state from which the quantization-aware stage of the shipped weights starts. Its sha256 `21a2de8c…` is the end of stage b3 in section 5.3 of `training/README.md` of the source tree (`https://github.com/marwoelk/cmix-neif-2.0`). The configuration of the run is `training/tf/3_train/chain/03_b3_bf16_epochs_30_to_400.json` of the same tree. b3 ran at a constant learning rate from epoch 30 on and is not cooled down. A test that continues this state should compare against a control run that continues it for the same number of epochs without the change. The checksums are in `SHA256SUMS`. The files are licensed under GPL-3, like the entry.