#3D-RoPE

'Uni-MoE-2.0-Omni is an open omnimodal MoE on a Qwen2.5-7B core that unifies text, image, audio and video via a shared token interface, dynamic expert routing and staged training for robust multimodal understanding and generation.'

'Zhipu AI released GLM-4.5V, an open-source vision-language model that combines a 106B parameter MoE backbone with 12B active parameters, 64K token context and a tunable Thinking Mode for advanced multimodal reasoning.'