add_batch
The functionadd_batch(batch, collection_name) uploads a batch of documents to a collection.
Input
batch – Represents the batch of documents to upload. The structure of each document must conform to the database schema configuration file. It must be of type list [dictionary].
collection_name – Specifies the name of the Collection into which to load the document.
Example 1 -
The following code snippet builds a list of documents in a temporary variable named batch and then uploads each batch using –
hyperspace_client.add_batch(batch, collection_name)hyperspaceClient.addBatch(batch, collection_name);await hyperspaceClient.addBatch(batch, collection_name);Response
{'code': 200, 'message': 'Batch successfully added', 'status': 'OK'}
Example 2 -
The following example uploads batches of 250 documents for. Documents are added to the batch, and once a batch reaches 250 documents, it's uploaded to the Hyperspace Collection.
Copy the following code snippet
BATCH_SIZE = 250
batch = []
collection_name = "new_collection"
for i, document in enumerate(documents):
batch.append(document )
if (i+1) % BATCH_SIZE == 0:
response = hyperspace_client.add_batch(batch, collection_name)
batch.clear()
if batch:
response = hyperspace_client.add_batch(batch, collection_name)
hyperspace_client.commit(collection_name)import java.util.ArrayList;
final int batchSize = 250;
for (int i= 0; index < documents.size(); i++) {
batch.add(documents.get(i));
if ((i+ 1) % batchSize == 0) {
List<DataPoint> batchCopy = new ArrayList<>(batch);
futures.add(hyperspaceClient.addBatch(batchCopy, collectionName));
batch.clear();
}
}
if (!batch.isEmpty()) {
futures.add(hyperspaceClient.addBatch(new ArrayList<>(batch), collectionName));
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
hyperspaceClient.commit(collectionName).join();const BATCH_SIZE = 250;
const batch: any[] = [];
const collectionName = "new_collection";
for (const [i, document] of documents.entries()) {
batch.push(document);
if ((i + 1) % BATCH_SIZE == 0) {
await client.addBatch(collectionName, batch);
batch = [];
}
}
if (batch.length != 0) {
await client.addBatch(collectionName, batch);
}
await client.commit(collectionName)Where –
document – Represents the document to upload. The structure of each document must be according to the database schema configuration file. Must be of type dictionary.
BATCH_SIZE – Specifies the number of documents in a batch.
commit – Is required for vector search only. commit should only be performed after the data upload is complete.
In this method, each document will be assigned with an automatic identifier.
Last updated