Interface VectorUtilSupport
public interface VectorUtilSupport
Interface for implementations of VectorUtil support.
- NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.
-
Method Summary
Modifier and TypeMethodDescriptionfloatcosine(byte[] a, byte[] b) Returns the cosine similarity between the two byte vectors.floatcosine(float[] v1, float[] v2) Returns the cosine similarity between the two vectors.intdotProduct(byte[] a, byte[] b) Returns the dot product computed over signed bytes.floatdotProduct(float[] a, float[] b) Calculates the dot product of the given float arrays.intfindNextGEQ(int[] buffer, int target, int from, int to) Given an arraybufferthat is sorted between indexes0inclusive andtoexclusive, find the first array index whose value is greater than or equal totarget.longint4BitDotProduct(byte[] int4Quantized, byte[] binaryQuantized) Compute the dot product between a quantized int4 vector and a binary quantized vector.intint4DotProduct(byte[] a, boolean apacked, byte[] b, boolean bpacked) Returns the dot product over the computed bytes, assuming the values are int4 encoded.floatminMaxScalarQuantize(float[] vector, byte[] dest, float scale, float alpha, float minQuantile, float maxQuantile) Quantizesvector, putting the result intodest.floatrecalculateScalarQuantizationOffset(byte[] vector, float oldAlpha, float oldMinQuantile, float scale, float alpha, float minQuantile, float maxQuantile) Recalculates the offset forvector.intsquareDistance(byte[] a, byte[] b) Returns the sum of squared differences of the two byte vectors.floatsquareDistance(float[] a, float[] b) Returns the sum of squared differences of the two vectors.
-
Method Details
-
dotProduct
float dotProduct(float[] a, float[] b) Calculates the dot product of the given float arrays. -
cosine
float cosine(float[] v1, float[] v2) Returns the cosine similarity between the two vectors. -
squareDistance
float squareDistance(float[] a, float[] b) Returns the sum of squared differences of the two vectors. -
dotProduct
int dotProduct(byte[] a, byte[] b) Returns the dot product computed over signed bytes. -
int4DotProduct
int int4DotProduct(byte[] a, boolean apacked, byte[] b, boolean bpacked) Returns the dot product over the computed bytes, assuming the values are int4 encoded. -
cosine
float cosine(byte[] a, byte[] b) Returns the cosine similarity between the two byte vectors. -
squareDistance
int squareDistance(byte[] a, byte[] b) Returns the sum of squared differences of the two byte vectors. -
findNextGEQ
int findNextGEQ(int[] buffer, int target, int from, int to) Given an arraybufferthat is sorted between indexes0inclusive andtoexclusive, find the first array index whose value is greater than or equal totarget. This index is guaranteed to be at leastfrom. If there is no such array index,tois returned. -
int4BitDotProduct
long int4BitDotProduct(byte[] int4Quantized, byte[] binaryQuantized) Compute the dot product between a quantized int4 vector and a binary quantized vector. It is assumed that the int4 quantized bits are packed in the byte array in the same way as theOptimizedScalarQuantizer.transposeHalfByte(byte[], byte[])and that the binary bits are packed the same way asOptimizedScalarQuantizer.packAsBinary(byte[], byte[]).- Parameters:
int4Quantized- half byte packed int4 quantized vectorbinaryQuantized- byte packed binary quantized vector- Returns:
- the dot product
-
minMaxScalarQuantize
float minMaxScalarQuantize(float[] vector, byte[] dest, float scale, float alpha, float minQuantile, float maxQuantile) Quantizesvector, putting the result intodest.- Parameters:
vector- the vector to quantizedest- the destination vectorscale- the scaling factoralpha- the alpha valueminQuantile- the lower quantile of the distributionmaxQuantile- the upper quantile of the distribution- Returns:
- the corrective offset that needs to be applied to the score
-
recalculateScalarQuantizationOffset
float recalculateScalarQuantizationOffset(byte[] vector, float oldAlpha, float oldMinQuantile, float scale, float alpha, float minQuantile, float maxQuantile) Recalculates the offset forvector.- Parameters:
vector- the vector to quantizeoldAlpha- the previous alpha valueoldMinQuantile- the previous lower quantilescale- the scaling factoralpha- the alpha valueminQuantile- the lower quantile of the distributionmaxQuantile- the upper quantile of the distribution- Returns:
- the new corrective offset
-