Fdb.Bulk
Namespace: FoundationDB.Client · class
Helper class for bulk operations
Methods
AggregateAsync
static Task<TAggregate> AggregateAsync<TSource, TAggregate>(IFdbDatabase db, IEnumerable<TSource> source, Func<TAggregate> localInit, Func<TSource[], BatchOperationContext, TAggregate, Task<TAggregate>> body, CancellationToken ct)
Execute a potentially long aggregation on batches of elements from a source sequence, using as many transactions as necessary, and automatically scaling the size of each batch to maximize the throughput.
db— Source databasesource— Source sequence that will be split into batch. The size of each batch will scale up and down automatically depending on the speed of executionlocalInit— Lambda function that is called once, and returns the initial state that will be passed to the first batchbody— Retry-able lambda function that receives a batch of elements from the source sequence, the current context, and the previous state. If the transaction expires while this lambda function is running, it will be automatically retried with a new transaction, and a smaller batch.ct— Token used to cancel the operation
Returns: Task that completes when all the elements of source have been processed, a non retry-able error occurs, or ct is triggered
static Task<TResult> AggregateAsync<TSource, TAggregate, TResult>(IFdbDatabase db, IEnumerable<TSource> source, Func<TAggregate> init, Func<TSource[], BatchOperationContext, TAggregate, Task<TAggregate>> body, Func<TAggregate, TResult> transform, CancellationToken ct)
Execute a potentially long aggregation on batches of elements from a source sequence, using as many transactions as necessary, and automatically scaling the size of each batch to maximize the throughput.
db— Source databasesource— Source sequence that will be split into batch. The size of each batch will scale up and down automatically depending on the speed of executioninit— Lambda function that is called once, and returns the initial state that will be passed to the first batchbody— Retryable lambda function that receives a batch of elements from the source sequence, the current context, and the previous state. If the transaction expires while this lambda function is running, it will be automatically retried with a new transaction, and a smaller batch.transform— Lambda function called with the aggregate returned by the last batch, and which will compute the final result of the operationct— Token used to cancel the operation
Returns: Task that completes when all the elements of source have been processed, a non-retryable error occurs, or ct is triggered
ExportAsync
static Task<long> ExportAsync(IFdbDatabase db, KeyRange range, Func<KeyValuePair<Slice, Slice>[], long, CancellationToken, Task> handler, CancellationToken ct)
Export the content of a potentially large range of keys defined by a pair of begin and end keys.
db— Database used for the operationrange— Pair of keys defining the start rangehandler— Lambda that will be called for each batch of data read from the database. The first argument is the array of ordered key/value pairs in the batch, taken from the same database snapshot. The second argument is the offset of the first item in the array, from the start of the range. The third argument is a token should be used by any async i/o performed by the lambda.ct— Token used to cancel the operation
Returns: Number of keys exported
This method cannot guarantee that all data will be read from the same snapshot of the database, which means that writes committed while the export is running may be seen partially. Only the items inside a single batch are guaranteed to be from the same snapshot of the database.
static Task<long> ExportAsync(IFdbDatabase db, ISubspaceLocation path, Func<KeyValuePair<Slice, Slice>[], IKeySubspace, long, CancellationToken, Task> handler, CancellationToken ct)
Export the content of a potentially large range of keys defined by a pair of selectors.
db— Database used for the operationpath— Path of the subspace to exporthandler— Lambda that will be called for each batch of data read from the database. The first argument is the array of ordered key/value pairs in the batch, taken from the same database snapshot. The second argument is the offset of the first item in the array, from the start of the range. The third argument is a token should be used by any async i/o performed by the lambda.ct— Token used to cancel the operation
Returns: Number of keys exported
This method cannot guarantee that all data will be read from the same snapshot of the database, which means that writes committed while the export is running may be seen partially. Only the items inside a single batch are guaranteed to be from the same snapshot of the database.
static Task<long> ExportAsync(IFdbDatabase db, Slice beginInclusive, Slice endExclusive, Func<KeyValuePair<Slice, Slice>[], long, CancellationToken, Task> handler, CancellationToken ct)
Export the content of a potentially large range of keys defined by a pair of begin and end keys.
db— Database used for the operationbeginInclusive— Key defining the start range (included)endExclusive— Key defining the end of the range (excluded)handler— Lambda that will be called for each batch of data read from the database. The first argument is the array of ordered key/value pairs in the batch, taken from the same database snapshot. The second argument is the offset of the first item in the array, from the start of the range. The third argument is a token should be used by any async i/o performed by the lambda.ct— Token used to cancel the operation
Returns: Number of keys exported
This method cannot guarantee that all data will be read from the same snapshot of the database, which means that writes committed while the export is running may be seen partially. Only the items inside a single batch are guaranteed to be from the same snapshot of the database.
static Task<long> ExportAsync(IFdbDatabase db, KeySelector begin, KeySelector end, Func<KeyValuePair<Slice, Slice>[], long, CancellationToken, Task> handler, CancellationToken ct)
Export the content of a potentially large range of keys defined by a pair of selectors.
db— Database used for the operationbegin— Selector defining the start of the range (included)end— Selector defining the end of the range (excluded)handler— Lambda that will be called for each batch of data read from the database. The first argument is the array of ordered key/value pairs in the batch, taken from the same database snapshot. The second argument is the offset of the first item in the array, from the start of the range. The third argument is a token should be used by any async i/o performed by the lambda.ct— Token used to cancel the operation
Returns: Number of keys exported
This method cannot guarantee that all data will be read from the same snapshot of the database, which means that writes committed while the export is running may be seen partially. Only the items inside a single batch are guaranteed to be from the same snapshot of the database.
ForEachAsync
static Task ForEachAsync<TSource>(IFdbDatabase db, IEnumerable<TSource> source, Func<TSource[], BatchOperationContext, Task> body, CancellationToken ct)
Execute a potentially long read-only operation on batches of elements from a source sequence, using as many transactions as necessary, and automatically scaling the size of each batch to maximize the throughput.
db— Source databasesource— Source sequence that will be split into batch. The size of each batch will scale up and down automatically depending on the speed of executionbody— Retry-able lambda function that receives a batch of elements from the source sequence, the current context, and the previous state. If the transaction expires while this lambda function is running, it will be automatically retried with a new transaction, and a smaller batch.ct— Token used to cancel the operation
Returns: Task that completes when all the elements of source have been processed, a non retry-able error occurs, or ct is triggered
static Task ForEachAsync<TSource>(IFdbDatabase db, IEnumerable<TSource> source, Action<TSource[], BatchOperationContext> body, CancellationToken ct)
Execute a potentially long read-only operation on batches of elements from a source sequence, using as many transactions as necessary, and automatically scaling the size of each batch to maximize the throughput.
db— Source databasesource— Source sequence that will be split into batch. The size of each batch will scale up and down automatically depending on the speed of executionbody— Retry-able lambda function that receives a batch of elements from the source sequence, the current context, and the previous state. If the transaction expires while this lambda function is running, it will be automatically retried with a new transaction, and a smaller batch.ct— Token used to cancel the operation
Returns: Task that completes when all the elements of source have been processed, a non retry-able error occurs, or ct is triggered
static Task ForEachAsync<TSource, TLocal>(IFdbDatabase db, IEnumerable<TSource> source, Func<TLocal> localInit, Func<TSource[], BatchOperationContext, TLocal, Task<TLocal>> body, Action<TLocal> localFinally, CancellationToken ct)
Execute a potentially long read-only operation on batches of elements from a source sequence, using as many transactions as necessary, and automatically scaling the size of each batch to maximize the throughput.
db— Source databasesource— Source sequence that will be split into batch. The size of each batch will scale up and down automatically depending on the speed of executionlocalInit— Lambda function that is called once, and returns the initial state that will be passed to the first batchbody— Retryable lambda function that receives a batch of elements from the source sequence, the current context, and the previous state. If the transaction expires while this lambda function is running, it will be automatically retried with a new transaction, and a smaller batch.localFinally— Lambda function that will be called after the last batch, and will be passed the last known state.ct— Token used to cancel the operation
Returns: Task that completes when all the elements of source have been processed, a non-retryable error occurs, or ct is triggered
static Task ForEachAsync<TSource, TLocal>(IFdbDatabase db, IEnumerable<TSource> source, Func<TLocal> localInit, Func<TSource[], BatchOperationContext, TLocal, TLocal> body, Action<TLocal> localFinally, CancellationToken ct)
Execute a potentially long read-only operation on batches of elements from a source sequence, using as many transactions as necessary, and automatically scaling the size of each batch to maximize the throughput.
db— Source databasesource— Source sequence that will be split into batch. The size of each batch will scale up and down automatically depending on the speed of executionlocalInit— Lambda function that is called once, and returns the initial state that will be passed to the first batchbody— Retry-able lambda function that receives a batch of elements from the source sequence, the current context, and the previous state. If the transaction expires while this lambda function is running, it will be automatically retried with a new transaction, and a smaller batch.localFinally— Lambda function that will be called after the last batch, and will be passed the last known state.ct— Token used to cancel the operation
Returns: Task that completes when all the elements of source have been processed, a non retry-able error occurs, or ct is triggered
InsertAsync
static Task<long> InsertAsync<T>(IFdbDatabase db, IEnumerable<T> source, Action<T, IFdbTransaction> handler, CancellationToken ct)
Inserts a potentially large sequence of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.ct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non-retryable error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
static Task<long> InsertAsync<T>(IFdbDatabase db, IEnumerable<T> source, Func<T, IFdbTransaction, Task> handler, CancellationToken ct)
Inserts a potentially large sequence of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.ct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non-retryable error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
static Task<long> InsertAsync<T>(IFdbDatabase db, IEnumerable<T> source, Action<T, IFdbTransaction> handler, WriteOptions options, CancellationToken ct)
Inserts a potentially large sequence of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.options— Custom options used to configure the behaviour of the operationct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non-retryable error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
static Task<long> InsertAsync<T>(IFdbDatabase db, IEnumerable<T> source, Func<T, IFdbTransaction, Task> handler, WriteOptions options, CancellationToken ct)
Inserts a potentially large sequence of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.options— Custom options used to configure the behaviour of the operationct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non-retryable error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
InsertBatchedAsync
static Task<long> InsertBatchedAsync<T>(IFdbDatabase db, IEnumerable<T> source, Action<T[], IFdbTransaction> handler, CancellationToken ct)
Inserts a potentially large number of batches of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.ct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non-retryable error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
static Task<long> InsertBatchedAsync<T>(IFdbDatabase db, IEnumerable<T> source, Func<T[], IFdbTransaction, Task> handler, CancellationToken ct)
Inserts a potentially large number of batches of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.ct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non retry-able error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
static Task<long> InsertBatchedAsync<T>(IFdbDatabase db, IEnumerable<T> source, Action<T[], IFdbTransaction> handler, WriteOptions options, CancellationToken ct)
Inserts a potentially large number of batches of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.options— Custom options used to configure the behaviour of the operationct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non retry-able error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
static Task<long> InsertBatchedAsync<T>(IFdbDatabase db, IEnumerable<T> source, Func<T[], IFdbTransaction, Task> handler, WriteOptions options, CancellationToken ct)
Inserts a potentially large number of batches of items into the database, by using as many transactions as necessary, and automatically retrying if needed.
db— Database used for the operationsource— Sequence of items to be processedhandler— Lambda called at least once for each item in the source. The method may not have any side effect outside the passed transaction.options— Custom options used to configure the behaviour of the operationct— Token used to cancel the operation
Returns: Number of items that have been inserted
In case of a non retry-able error, some of the items may remain in the database. Other transactions running at the same time may observe only a fraction of the items until the operation completes.
WriteAsync
static Task<long> WriteAsync(IFdbDatabase db, IEnumerable<KeyValuePair<Slice, Slice>> data, CancellationToken ct)
Writes a potentially large sequence of key/value pairs into the database, by using as many transactions as necessary, and automatically scaling the size of each batch.
db— Database used for the operationdata— Sequence of key/value pairsct— Token used to cancel the operation
Returns: Total number of values inserted in the database
In case of a non-retry-able error, some of the keys may remain in the database. Other transactions running at the same time may observe only a fraction of the keys until the operation completes.
static Task<long> WriteAsync(IFdbDatabase db, IEnumerable<(Slice, Slice)> data, CancellationToken ct)
Writes a potentially large sequence of key/value pairs into the database, by using as many transactions as necessary, and automatically scaling the size of each batch.
db— Database used for the operationdata— Sequence of key/value pairsct— Token used to cancel the operation
Returns: Total number of values inserted in the database
In case of a non retry-able error, some of the keys may remain in the database. Other transactions running at the same time may observe only a fraction of the keys until the operation completes.
static Task<long> WriteAsync(IFdbDatabase db, IEnumerable<KeyValuePair<Slice, Slice>> data, WriteOptions options, CancellationToken ct)
Writes a potentially large sequence of key/value pairs into the database, by using as many transactions as necessary, and automatically scaling the size of each batch.
db— Database used for the operationdata— Sequence of key/value pairsoptions— Custom options used to configure the behaviour of the operationct— Token used to cancel the operation
Returns: Total number of values inserted in the database
In case of a non retry-able error, some of the keys may remain in the database. Other transactions running at the same time may observe only a fraction of the keys until the operation completes.
static Task<long> WriteAsync(IFdbDatabase db, IEnumerable<(Slice, Slice)> data, WriteOptions options, CancellationToken ct)
Writes a potentially large sequence of key/value pairs into the database, by using as many transactions as necessary, and automatically scaling the size of each batch.
db— Database used for the operationdata— Sequence of key/value pairsoptions— Custom options used to configure the behaviour of the operationct— Token used to cancel the operation
Returns: Total number of values inserted in the database
In case of a non retry-able error, some of the keys may remain in the database. Other transactions running at the same time may observe only a fraction of the keys until the operation completes.