Deedle


Addressing

Namespace: Deedle

An Address value is used as an interface between vectors and indices. The index maps keys of various types to address, which is then used to get a value from the vector.

Here is a brief summary of what we assume (and don't assume) about addresses:

  • Address is int64 (although we might need to generalize this in the future)
  • Different data sources can use different addressing schemes (as long as both index and vector use the same scheme)
  • Addresses don't have to be continuous (e.g. if the source is partitioned, it can use 32bit partition index + 32bit offset in the partition)
  • In the in-memory representation, address is just index into an array
  • In the BigDeedle representation, address is abstracted and comes with AddressOperations that specifies how to use it (tests use linear offset and partitioned representation)

Nested types and modules

TypeDescription
Address

Address is int64<address>. We use unit of measure annotation to make sure that correct conversion functions are used.

IAddressOperations

Various implementations can use different schemes for working with addresses (for example, address can be just a global offset, or it can be pair of int32 values that store partition and offset in a partition). This interface represents a specific address range and abstracts operations that BigDeedle needs to perform on addresses (within the specified range)

IAddressingScheme

An empty interface that is used as an marker for "addressing schemes". As discussed above, Deedle can use different addressing schemes. We need to make sure that the index and vector share the scheme - this is done by attaching IAddressingScheme to each index or vector and checking that they match. Implementations must support equality!

LinearAddressingScheme

Represents a linear addressing scheme where the addresses are 0 .. <size>-1.

address
ModuleDescription
Address
LinearAddress

Address operations that are used by the standard in-memory Deedle structures (LinearIndex and ArrayVector). Here, address is a positive array offset.

Fork me on GitHub