llava
Category: multimodal-and-emerging · License: MIT · Version: 1.0.0
Tags: LLaVA, Vision-Language, Multimodal, Visual Question Answering, Image Chat, CLIP, Vicuna, Conversational AI, Instruction Tuning, VQA
When to use it
Large Language and Vision Assistant. Enables visual instruction tuning and image-based conversations. Combines CLIP vision encoder with Vicuna/LLaMA language models. Supports multi-turn image chat, visual question answering, and instruction following. Use for vision-language chatbots or image understanding tasks. Best for conversational image analysis.Example requests
Ask your agent in plain language, for example:- “How do I run LLaVA on a folder of microscopy screenshots and ask it questions about each one?”
- “I’m on a single 24 GB RTX 4090 and want to run llava-v1.5-13b on scanned lab notebook pages from notebooks_2024/. Loading it in fp16 gives ‘CUDA out of memory’. What should I change, and how well will it read the handwriting?”
- “I want an open-source assistant that can look at a chest X-ray figure from a paper and then answer follow-up questions about it over several turns, running locally. What do you suggest, and how would I set it up?”
Install
/plugin marketplace add KalarisLabs/research-agent-skills and install the whole multimodal-and-emerging category with /plugin install multimodal-and-emerging@research-agent-skills.