#Transformer

Explore Google's T5Gemma 2, an advanced encoder-decoder model family emphasizing multimodality and long context for developers.

IBM released Granite 4.0, a hybrid Mamba-2/Transformer LLM family that cuts serving memory by over 70% for long-context inference while keeping strong instruction-following and tool-use performance.

Falcon-H1 from TII introduces a hybrid model combining attention and state space mechanisms, achieving performance on par with leading 70B parameter LLMs while optimizing efficiency and scalability.

TII’s Falcon-H1 series introduces hybrid Transformer-SSM models that combine efficiency and performance, supporting long contexts, multilingual processing, and scalable deployment from 0.5B to 34B parameters.

Mila & Universite de Montreal researchers introduce FoX, a novel Transformer variant with learnable forget gates that improve long-context language modeling efficiency and accuracy without computational trade-offs.