4D AI: Sorting Query Results by Vector Similarity

In the Semantic search: querying by vector similarity post, we introduced how to query entities using vector similarity, a powerful way to find the most relevant records based on meaning rather than exact values.

4D 21 R2 comes with a new way to sort query results using 4D.Vector fields. You can not only filter entities by similarity but also sort your query results using the .query() function so that the most relevant and semantically closest entities appear first. Whether you’re building an intelligent search experience, a recommendation engine, or an AI-driven assistant, this new feature ensures that your results are ordered by relevance, providing more accurate and user-friendly outcomes.

Ordering results by similarity

As a 4D developer, you can now sort the results of a query based on vector similarity directly within your query expression. This enhancement applies to both 4D code and REST requests, making it simple to retrieve entities in order of semantic closeness.

When you execute a query on a vector field, and that same field appears in the order by clause, 4D automatically orders the results by their similarity (or distance) calculation.

var $result:=$clientAI.embeddings.create("A general manager living in France"; $model)
Var $inputEmbedding:=$result.vector
var $comparisonVector := {vector: $inputEmbedding; threshold: 0.4}

var $results := ds.MyTable.query("myVectorField >= :1 order by myVectorField"; $comparisonVector)

COMBINING SEMANTIC SORTING WITH CLASSICAL SORTING

A key advantage of 4D’s approach is that vector-based semantic ordering can be combined with traditional ORDA filters in the same query.

var $comparisonVector := {vector: $myVector; metric: mk cosine; threshold: 0.4} 
var $results := ds.MyTable.query("myVectorField <= :1 AND salary>100000 order by  myVectorField, salary desc"; $comparisonVector)

In this example, the query retrieves employees whose job descriptions are semantically close to the input text and whose salary exceeds 100,000, and orders them according to the vector similarity and the salary.

Using the REST API

This feature is also available through the REST API. You can sort your results by vector similarity with a simple query parameter:

GET /rest/MyTable/?$filter="myVectorField>=:1 order by myVectorField"&$params=[{"vector":[1,2,3,...],"threshold":0.4}]
// OR
GET /rest/MyTable/?$filter="myVectorField>=:1"&$params=[{"vector":[1,2,3,...],"threshold":0.4}]&$orderby="Vector"

Conclusion

This enhancement makes AI-powered queries even more practical and intuitive. You can now:

• Query your data using vector similarity.

• Sort the results by similarity, bringing the most relevant entities to the top.

• Use the feature consistently across both 4D and REST contexts.

This means you can easily build features like semantic search, recommendation lists, or context-aware lookups where the most similar results appear first.