Storage structure

Storage references the RoStore instance and its associated data.

When RoStore storage is created, some important constants that define it, like maximum total size and the block size are provided. These parameters can't be changed afterwards for this storage, as they define its operation and width of its space.

See mediaProperties here from POST /admin/store/create operation:

{
  "mediaProperties": {
    "maxTotalSize": 1073741824,
    "blockSize": 4096,
    "closeUnusedBlocksAfterMillis": 5000,
    "closeUnusedSequencesAfterMillis": 10000
  },
  "containerListProperties": {
    "autoCloseContainersAfterMillis": 60000,
    "cleanupIntervalMillis": 5000,
    "maxCleanupsPerCycle": 5,
    "maxContainersPerList": 1024,
    "maxContainersListSize": 5242880,
    "maxKeyOperationsPerShard": 10
  }
}

This operation is executed after the RoStore is installed.

The block size should be aligned with the size of the virtual memory page and can be just one virtual page or a multiple of them. Usually the size of the virtual memory page is 4KB (4096 bytes) in the modern PC-like machines. All these blocks are managed by the rostore media entity (Java object) and are mapped or unmapped from the memory based on the requests to this object.

Two other parametres from this section define the garbage collector behaviour of the RoStore, closeUnusedBlocksAfterMillis defines how fast the unused blocks should be unmapped, closeUnusedSequencesAfterMillis is used to garage-collect the sequences of blocks, which are special structures that manages the set of blocks, e.g. container's shards would be under control of such a sequence.

Containers

Containers a separately managed set of keys within the RoStore. For example, set of API Keys is stored separately from other keys, so they are stored in the special container. Containers are namespaces for keys, containers are also configured independently, and may have their own evicton policies, and so further. Generally, every container can be seen as an independent database. The second section from the json above defines how the list of containers is managed within the RoStore.

autoCloseContainersAfterMillis defines how fast the container-specific metadata should be removed from the memory if the container is idle for some time; cleanupIntervalMillis defines the time interval to cleanup the container keys. Clean up is needed to physically remove the expired keys. maxCleanupsPerCycle defines how many clean-ups the conatiner shards are allowed to execute per clean up cycle; maxContainersPerList specifies the total maximum number of containers in the storage, usually that number should not be too high. The architecture of the RoStore does not support too many containers; maxContainersListSize specifies the total size of all container names together, this is only needed to prevent a memory misusage; maxKeyOperationsPerShard specifies how many active operations a container shard can maintain in parallel.

For the access permissions on container level please refer here.

To create a new container on the RoStore service the POST /admin/container/{container} can be executed:

{
  "maxSize": 0,
  "shardNumber": 5,
  "maxTTL": 0
}

maxSize specifies the maximum size of the container in bytes. If this parameter left as 0 the container would not have any size restrictions; shardNumber specifies the number of shards in the container, it is recommended to have 1 to 20 shards. Every shard is a complex structure and its usage should be limited if memory usage of any concern. About shards read the next section; maxTTL specifies the maximum life span of the keys in seconds within the container, if it is left 0, the key will have any expiration time.

Container shards

Every container is split into one or several shards. The shards are indepentent parts of a container. Every shard can support many read operations (see maxKeyOperationsPerShard above), but only one write (or delete) operation. Shard is blocked by the write operation until it is executed completely. Here the write operation is meant to refer the key-write operation. For the value-write operation the shard is not blocked. Let's take a look how the key-value write operation is executed in the RoStore service. First, the value is stored to the storage, this operation can take quite a time depending on the value data size, connectivity latency between client and the rostore service and so on. This operation is non-blocking for any other operation on the storage, as it happens independently from the rest. After it is finished, the RoStore calculates a hash code of the key and finds a shard for this key. If any read operation from the shard is on-going, the key-write operation is put to the queue and will wait until all the read operations are finished. If any read operation would come after the write operation is arrived and queued, they all will also be put to the queue. Once, all the on-going read operations are finished, the write operation is taken from the queue and blocks the shard until it is executed. After write operation is finshed, the queue is processed further. In this description, the shard is waiting for the on-going read operations to be finished. It only waits for the key-read operations and not the associated value-read operations. As they will be executed in a separate and independent maner and not block the shard.

Here is major outcome is that the keys should preferbly be short, maybe some kilobytes, where is the value can be arbitrary long, which won't have any influence on the storage performance.

Containers​

Container shards​

Containers

Container shards