Table of Contents

Class RowGroupWriter

Namespace
ParquetSharp
Assembly
ParquetSharp.dll

Writes a row group to a Parquet file. Allows access to individual ColumnWriter instances for each column in the row group.

public sealed class RowGroupWriter : IDisposable
Inheritance
RowGroupWriter
Implements
Inherited Members

Properties

Buffered

public bool Buffered { get; }

Property Value

bool

CurrentColumn

public int CurrentColumn { get; }

Property Value

int

EstimatedBufferedStats

Estimated sizes of data buffered by the column writers (levels, values and dictionaries) that has not yet been written to pages.

public BufferedWriterStats EstimatedBufferedStats { get; }

Property Value

BufferedWriterStats

Remarks

For a non-buffered row group, this only includes data buffered for the current column. Once the row group is closed, all fields are zero.

This does not include completed data pages held in memory while a column is dictionary encoded, which are reported by TotalCompressedBytes. See BufferedWriterStats for details.

NumColumns

public int NumColumns { get; }

Property Value

int

NumRows

public long NumRows { get; }

Property Value

long

TotalBytesWritten

public long TotalBytesWritten { get; }

Property Value

long

TotalCompressedBytes

Total compressed bytes of data pages that have been completed but not yet written by the page writers.

public long TotalCompressedBytes { get; }

Property Value

long

Remarks

Data pages are held in memory rather than written while a column is dictionary encoded, until the dictionary page is written when the column is closed. This is an estimate, as the size of each page header is over-estimated.

TotalCompressedBytesWritten

Total compressed bytes written by the page writers so far, including data and dictionary pages.

public long TotalCompressedBytesWritten { get; }

Property Value

long

Remarks

For a non-buffered row group, while the row group is open this only includes the bytes written for the current column. After calling Close(), it includes all columns.

While a column is dictionary encoded, completed data pages are held in memory until the dictionary page is written, and are counted in TotalCompressedBytes rather than here. See BufferedWriterStats for how to estimate the total size of a buffered row group.

Methods

Close()

public void Close()

Column(int)

Get the column writer for the i-th column.

public ColumnWriter Column(int i)

Parameters

i int

The index of the column.

Returns

ColumnWriter

A ColumnWriter for the i-th column.

Remarks

This method may only be used with a buffered row group writer, created with AppendBufferedRowGroup().

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

public void Dispose()

NextColumn()

Get the column writer for the next column.

public ColumnWriter NextColumn()

Returns

ColumnWriter

A ColumnWriter for the next column.

Remarks

This method is not valid when using a buffered row group writer.