phoenix-observability
Category: ml-inference-and-ops · License: MIT · Version: 1.0.0
Tags: Observability, Phoenix, Arize, Tracing, Evaluation, Monitoring, LLM Ops, OpenTelemetry
When to use it
Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.Example requests
Ask your agent in plain language, for example:- “How do I get Phoenix tracing running locally for my OpenAI calls?”
- “I’m running a RAG pipeline in LlamaIndex over our lab’s protocol PDFs. The answers are sometimes wrong and I can’t tell if retrieval or generation is at fault. I set up Phoenix at localhost:6006 but my project shows no traces, only an empty ‘default’ project. How do I fix that and then use the traces to find the failing step?”
- “I have 200 question-answer pairs from our clinical-trial summarization model and want to systematically score hallucination and relevance with an LLM judge, compare two prompt versions, and keep the test set versioned so I can rerun it in CI. Self-hosted only, no SaaS. What setup do you recommend?”
Install
/plugin marketplace add KalarisLabs/research-agent-skills and install the whole ml-inference-and-ops category with /plugin install ml-inference-and-ops@research-agent-skills.
Related
evaluating-cosmos-policylangsmith-observabilityspeculative-decoding- Skills for machine learning research