Class RowGroupWriter
- Namespace
- ParquetSharp
- Assembly
- ParquetSharp.dll
Writes a row group to a Parquet file. Allows access to individual ColumnWriter instances for each column in the row group.
public sealed class RowGroupWriter : IDisposable
- Inheritance
-
RowGroupWriter
- Implements
- Inherited Members
Properties
Buffered
public bool Buffered { get; }
Property Value
CurrentColumn
public int CurrentColumn { get; }
Property Value
EstimatedBufferedStats
Estimated sizes of data buffered by the column writers (levels, values and dictionaries) that has not yet been written to pages.
public BufferedWriterStats EstimatedBufferedStats { get; }
Property Value
Remarks
For a non-buffered row group, this only includes data buffered for the current column. Once the row group is closed, all fields are zero.
This does not include completed data pages held in memory while a column is dictionary encoded, which are reported by TotalCompressedBytes. See BufferedWriterStats for details.
NumColumns
public int NumColumns { get; }
Property Value
NumRows
public long NumRows { get; }
Property Value
TotalBytesWritten
public long TotalBytesWritten { get; }
Property Value
TotalCompressedBytes
Total compressed bytes of data pages that have been completed but not yet written by the page writers.
public long TotalCompressedBytes { get; }
Property Value
Remarks
Data pages are held in memory rather than written while a column is dictionary encoded, until the dictionary page is written when the column is closed. This is an estimate, as the size of each page header is over-estimated.
TotalCompressedBytesWritten
Total compressed bytes written by the page writers so far, including data and dictionary pages.
public long TotalCompressedBytesWritten { get; }
Property Value
Remarks
For a non-buffered row group, while the row group is open this only includes the bytes written for the current column. After calling Close(), it includes all columns.
While a column is dictionary encoded, completed data pages are held in memory until the dictionary page is written, and are counted in TotalCompressedBytes rather than here. See BufferedWriterStats for how to estimate the total size of a buffered row group.
Methods
Close()
public void Close()
Column(int)
Get the column writer for the i-th column.
public ColumnWriter Column(int i)
Parameters
iintThe index of the column.
Returns
- ColumnWriter
A ColumnWriter for the i-th column.
Remarks
This method may only be used with a buffered row group writer, created with AppendBufferedRowGroup().
Dispose()
Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
public void Dispose()
NextColumn()
Get the column writer for the next column.
public ColumnWriter NextColumn()
Returns
- ColumnWriter
A ColumnWriter for the next column.
Remarks
This method is not valid when using a buffered row group writer.