Links
Comment on page

Candidate Score

Hyperspace support various methods of scoring and arithmetics, based on rarity of keywords in the collection.

Rarity Score (TF-IDF)

The rarity score can be calculated for matched keywords. Hyperspace calculate this score over keywords or lists of keywords, using the TF-IDF formula.
Two different types of usages are currently allowed -
  • rarity_max(str fieldname) returns the maximum rarity out of all the keywords in the list,
  • rarity_sum(str fieldname) returns the sum of rarities of all the keywords in the list.
For keyword fields (non lists) the two functions will return the same result.
Example:
score = rarity_max("cities") + rarity_sum("streets")

Score Operations

Hyperspace allows multiple methods for score arithmetic, as explained below
  • Sum
  • Max
  • Arithmetic operations

Sum of Scores

The function receives n scores (results of score functions) and returns their sum
Syntax
sum (float score1, float score2,...)
Example
score1 = rarity_max("city")
score2 = rarity_max("Country")
score3 = rarity_max("Continent")
score4 = .....
score_sum = sum(score1, score2, score3...)
Where -
  • score1, score2, score3 are the results of a score function.
  • score_sum is the sum of score1, score2, score3...

Max of Scores

The function receives n scores (results of score functions) and returns the maximum of their values
Syntax
max(float score1, float score2)
Example
score1 = rarity_max("city")
score2 = rarity_max("Country")
score3 = rarity_max("Continent")
score4 = .....
score_max = max(score1, score2, score3...)
Where -
  • score1, score2, score3 are the results of a score function.
  • score_max is the maximum between score1, score2, score3...
rarity_sum and rarity_max may only return different score for list[keywords]. In particular, when used for matching fields of type keyword, they will always return the same score.

Arithmetic Operators

Hyperspace allows arithmetic operations between scores, using the operators +, *, -, / . These operators can be used in combination with the operator =
Example
score0 = 0.0
if (match("field 1") or match("field2") or match("Expiration date")):
score0 += rarity_max("visit_times_in_personal care")
score0 -= rarity_sum("Credit card")
score1 = 2 * score0
Where-
  • score0 is the result of a score function.

Vector Distance

Hyperspace allows to include the KNN vector score in the lexical score function, by using the function distance(str vector_fieldname1, str vector_fieldname2, r32 min_score).
The distance()function calculates the KNN score based on the metric defined in the data configuration schema file. It will then return the score if it is above the min_score_threhold,or 0 otherwise min_score can be a dynamic value, provided as part of the query params.
By default,vector_fieldname2= vector_fieldname1 and min_score_threhold = 0

Limitations

The distance function can only be used as part of the last return statement.
In addition, all other return statements mustreturn 0, False or none. For example:
Example 1:
def score_function(params, doc):
if match("genre"):
return
else if match("countries"):
return False
score = rarity_max("tags")
if score < 1:
return 0
return score1 + 0.3 * distance("tagline_embedding", 0.2)
In the above example, distance calculates the KNN score between params["tagline_embedding"]and doc["tagline_embedding"]. If the score is above 0.2, the function will return score1 + 0.3 * knn_score. Otherwise it will return score1.
Example 2:
def score_function(params, doc):
score1 = rarity_max("tags")
return score1 + distance("tagline_embedding", "overview_embedding", params["min_score"])
In the above example, distance calculates the KNN score between params["tagline_embedding"]and doc["overview_embedding"]. If the score is above params["min_score"], it will return score1 + distance. Otherwise it will return score1.