Gregius Data – Vectors and Embeddings

Overview

Gregius Data’s vector subsystem generates and manages semantic embeddings that power AI search and retrieval across your WordPress content. It supports three vector generation approaches — internal TF-IDF with vocabulary management, a stateless HashingTF method requiring no preparation step, and provider-backed API embeddings — each scoped to a data connection and its active models.

Prerequisites

• WordPress 6.9+ with Gregius Data plugin installed and activated
• At least one AI model registered and active on the site
• At least one data connection configured and active

---

Understanding Vectors and Embeddings

Vectors are numerical representations of your site content that allow AI models to understand meaning and relevance. The subsystem converts your cleaned and chunked content — titles, excerpts, and content sections — into vector embeddings stored in dedicated PostgreSQL tables.

Three vector generation approaches are available:

Approach	Preparation needed	Use case
TF-IDF	Vocabulary must be prepared first	Corpus-aware search with term frequency weighting
HashingTF (Murmur3)	None — runs immediately	Zero-dependency baseline, no preparation step required
API embeddings	Provider model configuration	External AI provider embeddings (OpenAI, etc.)

Each model type stores vectors in its own table. TF-IDF uses a vocabulary corpus; HashingTF uses feature hashing with no vocabulary dependency; API embeddings rely on external provider services.

---

How to: Manage Vocabulary

TF-IDF generation requires a vocabulary — a corpus of terms and their frequency data — built from your site’s cleaned content. The vocabulary must be prepared before TF-IDF vectors can be generated for a connection.

Prepare vocabulary

When you prepare vocabulary, the system reads your site’s cleaned content corpus and builds the term frequency data required for TF-IDF vector generation. Preparation is scoped to a connection.

Check vocabulary status

You can inspect the current state of your vocabulary at any time. Status information includes:

• Version – The current vocabulary build version
• Post count context – How many posts the vocabulary was built from
• Generation timestamp – When the vocabulary was last rebuilt
• Drift indicators – Whether the vocabulary is out of date relative to current content

Clear vocabulary cache

When your site content changes significantly, you can clear the vocabulary cache and trigger a rebuild. This ensures TF-IDF vectors reflect the latest content state.

Tips

• Prepare vocabulary before running TF-IDF generation — generation will refuse to run if vocabulary is not ready
• Check vocabulary status if TF-IDF output quality seems degraded; stale vocabularies may reduce relevance
• Connection type (direct PostgreSQL or PostgREST-compatible) is detected automatically

---

How to: Generate Vectors

Vector generation converts your site content into searchable embeddings. You control when and how generation runs, which models to use, and which connection to target.

Generate vectors for a connection

Each connection has its own active models and content pipeline. When you trigger generation, the system:

• Resolves the model configuration for the connection
• Selects the matching strategy (TF-IDF, HashingTF, or API embeddings)
• Processes your cleaned and chunked content through that strategy
• Stores the resulting embeddings in the model’s vector table

Monitor generation progress

You can inspect:

• Queue – Pending generation items per connection
• Status – Overall generation health and recent outcomes
• Posts list – Per-post vector coverage and any failures

Tips

• Verify your data connection and model are active before generating
• HashingTF requires no vocabulary preparation — you can generate immediately
• API embeddings generation depends on external provider availability and rate limits

---

How to: Manage Connection-Model Associations

Each data connection can have its own set of active models. This lets you choose which embedding approach applies to which data source.

View active models

For each connection, you can see which models are currently associated. Each entry shows:

• Model ID – Unique identifier
• Provider – Where the model comes from (internal or external provider)
• Type – What the model is used for
• Status – Whether the model is active

Add or remove models

You can add a model to a connection or remove an existing association without touching plugin configuration files. When no connection-specific model is found, the system can fall back to the global model registry.

Tips

• Models registered in the global registry are available for assignment to connections
• Removing a model from a connection does not delete its vector data — manage vectors separately
• API key material is masked in all response payloads for security

---

How to: Remove Vectors

You can remove vector embeddings from your database. Two approaches are available depending on your dataset size.

Delete all vectors

For smaller datasets, you can clear all vectors for a connection, optionally scoped to a specific model. This removes all embedding rows from the model’s vector table.

Batch delete for large datasets

For large vector tables where a single delete request could time out, use batch delete. It removes vectors in manageable batches and reports progress after each step:

• Deleted – Rows removed in this batch
• Total deleted – Cumulative rows removed
• Has more – Whether remaining rows exist
• Duration – Time taken for the batch

The system handles provider-specific execution paths automatically — both direct PostgreSQL and PostgREST-compatible connections are supported.

Tips

• Batch delete is designed for large datasets; use the direct delete for smaller cleanup tasks
• If batch delete stops before completion, re-run to continue from remaining rows
• Deleting vectors does not affect your original content or cleaned/chunked data

---

Permissions

Area	Route	Who can use it
Vocabulary management	Vocabulary prepare, status, cache clear	Administrators only
Vector generation and queue	Generate, status, queue, posts list	Administrators only
Vector deletion	Delete all, batch delete	Administrators only
Connection-model associations	View, add, remove models per connection	Administrators only

---