Perfecting Merge-kit MoE’s: We don't need Mixtral-instructs “secret sauce” A write up by: Rombodawg Introduction: (Note: Please stop suggesting edits, Comments enabled for AI merging using MergKit, MoE creation, and paper feedback.) First of all I’d like to point out the elephant in the room… ...