Florence-2

The basic caption variant generates a concise, single-sentence summary identifying only the primary subject, making it ideal for high-speed indexing and basic alt-text. The detailed caption variant expands on this by incorporating spatial context and immediate environmental relationships while maintaining a compact single-sentence structure.

Select the more-detailed caption variant when you need dense, multi-sentence synthetic annotations for generative model training datasets like FLUX or SDXL LoRAs. It performs an exhaustive cataloging of background elements, surface textures, and fine spatial arrangements, whereas the standard detailed variant focuses primarily on foreground spatial relationships.

Florence-2 excels at photorealistic scene analysis, accessibility alt-text generation, large-scale search indexing, and automated LoRA dataset captioning. It is less suited for open-ended conversational visual reasoning, complex stylized 2D illustrations, anime aesthetic tagging, or transcribing stylized handwritten text.

Florence-2 was developed by Microsoft and released open-source in June 2024. Despite its compact footprint of 0.77 billion parameters, it leverages a sequence-to-sequence transformer architecture combining a DaViT vision encoder with a BART text decoder trained on the FLD-5B dataset containing 5.4 billion visual annotations.

All captioning variants—basic, detailed, and more-detailed—operate at a flat compute cost of 3.5 per execution, providing an economical solution for massive dataset processing and automated asset tagging.