Vector Databases 101: Giving AI Long-Term Memory

The Goldfish Memory Problem

Standard LLMs like GPT-4 have a "context window." They can remember the current conversation up to a limit (e.g., 128k tokens). But once you close the tab, that memory is gone. Furthermore, you cannot fit your entire company wiki into that window.

Enter Embeddings

To solve this, we convert text into vectors—long lists of numbers (coordinates) that represent the semantic meaning of the text.

"The cat sat on the mat" -> [0.002, -0.15, 0.82, ...]

If we also embed "The feline rested on the rug," the numbers will be very similar. Mathematically, these two vectors are "close" to each other in multi-dimensional space.

Vector Databases

A Vector IDB (like Pinecone, Milvus, or pgvector) is optimized to store and search these vectors. Unlike a SQL database looking for exact keyword matches, a Vector DB looks for nearest neighbors.

Why it matters for business

This technology powers:

• Semantic Search: Customers can search "shoe for running in rain" and find waterproof sneakers, even if the description doesn't have those exact keywords.
• Recommendation Engines: "Users who liked this article also liked..." based on semantic similarity of the content.
• Long-term Memory: An AI agent can store every interaction it has ever had with a user, retrieving relevant past memories only when needed.