Photo by Yogas Design on Unsplash

How Vespa works?

Wolf
3 min readAug 9, 2023

--

Vespa is an open-source, high-performance big data processing and serving engine developed by Yahoo and later open-sourced by Verizon Media. It is designed to handle large-scale, real-time data processing and serving for various applications such as search, recommendation, personalization, and more. Vespa is particularly well-suited for scenarios that require low-latency and high-throughput data access.

How Vespa works internally?

Internally, Vespa employs a distributed architecture that involves several key components working together:

Container Nodes and Services

  • Container Nodes: These nodes host Vespa application containers.
  • Application Containers: These are hosted on container nodes and provide various services like query processing, ranking, and document processing.
  • Query Processing: Containers process incoming user queries, applying ranking functions and retrieving relevant documents from the indexes.
  • Document Processing: Containers handle document ingestion, updating, and indexing.
  • Application Package: This is the configuration that defines the data schema, indexing, ranking, and other settings for the specific use case. It’s defined using a domain-specific language (DSL) that Vespa understands.

--

--

Wolf

A software engineer looking for learning more and share my knowledge.