ai database infrastructure
This page is about AI and LLM with respect to a database infrastructure.
1. Vector Databases
Vector databases are core to LLM retrieval and inference systems. They store embeddings and perform approximate nearest neighbor (ANN) searches.
- Pinecone – Serverless vector DB optimized for LLMs
- Weaviate – Open-source with semantic and hybrid search
- FAISS – Library for similarity search; not a full DB
- Milvus – Distributed, high-performance vector storage
- Qdrant – Open-source with production-ready APIs
- ChromaDB – Lightweight and ideal for prototyping
- Redis (vector extension) – Real-time vector search with hybrid capabilities
2. Relational Databases
Used for structured data like logs, metadata, and fine-tuning datasets.
- PostgreSQL
- Rich in features like JSONB, full-text search, and
pgvectorsupport - MySQL
- MariaDB – Less common now, still used in legacy apps
- SQLite – Embedded use in lightweight or local LLM tools
3. Document Stores
Useful for managing prompts, configuration, and LLM session context.
- MongoDB – Flexible for prompt stores and user session logs
- CouchDB
- Couchbase – Suited for distributed, offline-first AI apps
4. Time-Series & Logging Databases
Track usage, monitor performance, and support RLHF feedback loops.
- InfluxDB – Time-series data for model behavior monitoring
- Prometheus
- Grafana – Used for operational observability
- ClickHouse – High-speed analytics on inference logs
5. Data Lakes & Warehouses
Handle the bulk of pre-training data and feature store infrastructure.
- Databricks (Delta Lake) – Unified batch and stream storage for ML
- Snowflake – Cloud-native SQL warehouse supporting LLM features
- BigQuery – Google's scalable ML query backend
- Amazon S3 + Athena / Redshift – Core for training data pipelines
6. Graph Databases
Enable reasoning, symbolic augmentation, and knowledge graphs in LLM apps.
- Neo4j – Rich knowledge graphs with Cypher query language
- TigerGraph – Designed for real-time graph querying at scale
- Amazon Neptune – Graph DB integrated with AWS AI services
7. Hybrid Systems
Combining keyword, semantic, and embedding search in AI apps.
- LanceDB – Native support for LLM workflows
- PostgreSQL + pgvector + LangChain – Popular open-source RAG stack
- ElasticSearch
- kNN plugin – Combines dense and sparse retrieval
- Typesense – Fast, LLM-integrated search engine
Infrastructure Summary Table
| Role | Common Technologies |
| Vector Search | Pinecone, FAISS, Weaviate, Qdrant, Milvus |
| Structured Data | PostgreSQL, MySQL, SQLite |
| Unstructured Data | MongoDB, Couchbase |
| Monitoring & Logs | InfluxDB, ClickHouse, Prometheus |
| Training Storage | Databricks, Snowflake, BigQuery |
| Knowledge Graphs | Neo4j, TigerGraph, Neptune |
Trends & Takeaways
- PostgreSQL is emerging as a universal choice due to support for both structured and vector queries.
- Vector-native databases are now critical for retrieval-augmented generation (RAG).
- Hybrid search models (sparse + dense) are becoming standard in LLM toolchains.
- Graph DBs are resurging in agent memory and reasoning systems.