sparrow_ipc::utils Namespace Reference

Functions
std::pair< std::uint8_t *, int64_t >	get_bitmap_pointer_and_null_count (std::span< const uint8_t > validity_buffer_span, const int64_t length)
	Extracts bitmap pointer and null count from a validity buffer span.
std::span< const uint8_t >	get_buffer (const org::apache::arrow::flatbuf::RecordBatch &record_batch, std::span< const uint8_t > body, size_t &buffer_index)
	Extracts a buffer from a RecordBatch's body.
std::variant< std::vector< std::uint8_t >, std::span< const std::uint8_t > >	get_decompressed_buffer (std::span< const uint8_t > buffer_span, const org::apache::arrow::flatbuf::BodyCompression *compression)
	Retrieves a decompressed buffer or a view of the original buffer.
SPARROW_IPC_API size_t	align_to_8 (const size_t n)
template<std::ranges::input_range R> requires std::same_as<std::ranges::range_value_t<R>, sparrow::record_batch>
bool	check_record_batches_consistency (const R &record_batches)
	Checks if all record batches in a collection have consistent structure.
std::optional< int32_t >	parse_format (std::string_view format_str, std::string_view sep)

Function Documentation

align_to_8()

SPARROW_IPC_API size_t sparrow_ipc::utils::align_to_8

(

const size_t

n

)

Here is the caller graph for this function:

check_record_batches_consistency()

template<std::ranges::input_range R>
requires std::same_as<std::ranges::range_value_t<R>, sparrow::record_batch>

bool sparrow_ipc::utils::check_record_batches_consistency

(

const R &

record_batches

)

Checks if all record batches in a collection have consistent structure.

This function verifies that all record batches in the provided collection have:

• The same number of columns
• Matching data types for corresponding columns (same column index)

Template Parameters

R	Container type that holds sparrow::record_batch objects

Parameters

record_batches

Collection of record batches to check for consistency

Returns

true if all record batches have consistent structure or if the collection is empty, false if any structural inconsistencies are found

Note

An empty collection is considered consistent and returns true

The number of rows per record batch is not required to be the same

Definition at line 33 of file utils.hpp.

Here is the caller graph for this function:

get_bitmap_pointer_and_null_count()

std::pair< std::uint8_t *, int64_t > sparrow_ipc::utils::get_bitmap_pointer_and_null_count ( std::span< const uint8_t > validity_buffer_span, const int64_t length )

nodiscard

Extracts bitmap pointer and null count from a validity buffer span.

This function calculates the number of null values represented by the bitmap.

Parameters

validity_buffer_span	The validity buffer as a byte span.
length	The Arrow RecordBatch length (number of values in the array).

Returns

A pair containing:

• First: Pointer to the bitmap data (nullptr if buffer is empty)
• Second: Count of null values in the bitmap (0 if buffer is empty)

Note

If the bitmap buffer is empty, returns {nullptr, 0}

The returned pointer is a non-const cast of the original const data

Here is the caller graph for this function:

get_buffer()

std::span< const uint8_t > sparrow_ipc::utils::get_buffer ( const org::apache::arrow::flatbuf::RecordBatch & record_batch, std::span< const uint8_t > body, size_t & buffer_index )

nodiscard

Extracts a buffer from a RecordBatch's body.

This function retrieves a buffer span from the specified index in the RecordBatch's buffer list and increments the index.

Parameters

record_batch	The Arrow RecordBatch containing buffer metadata.
body	The raw buffer data as a byte span.
buffer_index	The index of the buffer to retrieve. This value is incremented by the function.

Returns

A std::span<const uint8_t> viewing the extracted buffer data.

Exceptions

std::runtime_error

if the buffer metadata indicates a buffer that exceeds the body size.

Here is the caller graph for this function:

get_decompressed_buffer()

std::variant< std::vector< std::uint8_t >, std::span< const std::uint8_t > > sparrow_ipc::utils::get_decompressed_buffer ( std::span< const uint8_t > buffer_span, const org::apache::arrow::flatbuf::BodyCompression * compression )

nodiscard

Retrieves a decompressed buffer or a view of the original buffer.

This function either decompresses the provided buffer span, if compression is specified, or returns a view of the original buffer without modification.

Parameters

buffer_span	A span of raw buffer data to be decompressed, or returned as-is if no decompression is needed.
compression	The compression algorithm to use. If nullptr, no decompression is performed.

Returns

A std::variant containing either:

• A std::vector<std::uint8_t> with the decompressed data, or
• A std::span<const std::uint8_t> providing a view of the original buffer_span if no decompression occurred.

Here is the caller graph for this function:

parse_format()

std::optional< int32_t > sparrow_ipc::utils::parse_format	(	std::string_view	format_str,
		std::string_view	sep )