Florence-2

The Basic variant produces a single concise sentence focusing strictly on the primary subject and action, making it ideal for rapid indexing and web alt-text. The Detailed variant provides a balanced description identifying main subjects, secondary props, and environmental landmarks for classic CLIP pipelines. The More Detailed variant outputs an exhaustive paragraph cataloging micro-textures, clothing materials, lighting conditions, and spatial arrangements for deep captioning.

Use the More Detailed variant when building datasets for FLUX and other architectures with T5-XXL text encoders, as they benefit from long natural-language descriptions of lighting, materials, and composition. For classic CLIP-based pipelines like SD 1.5 or SDXL, the Detailed variant is often preferred because it captures the core subject and setting without overwhelming the token limit with granular background descriptions.

Florence-2 is best suited for automated dataset tagging, reverse prompt engineering for generative image models, detailed alt-text generation, and objective visual attribute profiling. Its structured, matter-of-fact text outputs excel at describing single unified scenes cleanly without conversational filler.

Florence-2 struggles with complex spatial bounding coordinates, OCR transcription on heavily distorted or low-resolution embedded text, and multi-turn conversational reasoning. It can also produce occasional chromatic misidentifications in deep shadow areas or repeat descriptive phrases on dense repetitive textures when running in More Detailed mode.

Florence-2 was developed by Microsoft and released in June 2024 under the open-source MIT license. It features a compact 770-million-parameter sequence-to-sequence architecture combining a Document-aware Vision Transformer (DaViT) encoder with a BART-like text decoder, trained on the massive FLD-5B dataset containing 5.4 billion annotations across 126 million images.