Tutorial

Streaming Records

Partial Access is not supported if Transactions are enabled!

In some scenarios, inserting or looking up a record in one single operation may be difficult - i.e. if the record is too big. For such cases applications can use “Partial Access” to a record. This can be used to read and write several chunks of a huge record, or to “stream” data from or to the Database.

With Partial Access, records can be read or written in several smaller steps. If the flag UPS_PARTIAL is specified in ups_db_insert or ups_db_find (or one of the corresponding cursor functions), then upscaledb will evaluate the fields record->partial_offset and record->partial_size.

When writing a partial record, upscaledb will write record->partial_size bytes of the record data at offset record->partial_offset. The full record size will always be given in record->size. When a record is overwritten (see UPS_OVERWRITE) with a different record->size, the record will grow or shrink.

When writing with an invalid combination (the sum of partial_offset + partial_size exceeds the total record size), upscaledb will return UPS_INV_PARAMETER.

When reading a partial record, upscaledb will read record->partial_size bytes of the record data starting at offset record->partial_offset. The data will be stored in record->data. The size of the retrieved data is stored in record->size.

When reading a part of a record, and the partial_offset exceeds the total record size, upscaledb will return UPS_INV_PARAMETER. If the sum of partial_offset + partial_size exceeds the total record size, upscaledb will limit the partial_size and return the read size in record->size.

Partial access has a few limitations: it is not allowed in combination with Transactions and it only works with all records that are larger than 8 bytes. upscaledb has a very efficient and compact storage for small records with a size <= 8 bytes. This storage does not support partial access. The API therefore returns UPS_INV_PARAMETER if the record->size <= 8 and the flag UPS_PARTIAL is set.