https://jaypolra.github.io/tags/grad-cam/Grad-CAMHugo Blox Builder (https://hugoblox.com)en-usWed, 01 Jan 2025 00:00:00 +0000https://jaypolra.github.io/media/icon_hu_3943bc2ea2cd9fa1.pnghttps://jaypolra.github.io/tags/grad-cam/https://jaypolra.github.io/project/vlm-explainer/Wed, 01 Jan 2025 00:00:00 +0000https://jaypolra.github.io/project/vlm-explainer/<p>BLIP generates fluent captions but gives no account of why. The question this project answers: do the image regions the model highlights actually cause the caption, or do they just correlate with it?</p> <p>Built Grad-CAM token attribution to identify which image regions influenced specific caption words. Validated through perturbation testing: masking the dog region changed &ldquo;a family walking with their dog&rdquo; to &ldquo;a family walking on the beach.&rdquo; Causal influence, not correlation, is the standard.</p> <p>Added CLIP alignment scoring to cross-verify image-text grounding. Interactive Streamlit interface with 2-click region masking and layer-evolution visualization across shallow-to-deep BLIP representations.</p>