Class ColumnWriter
- Namespace
- ParquetSharp
- Assembly
- ParquetSharp.dll
Writer of physical Parquet values to a single column.
public abstract class ColumnWriter : IDisposable
- Inheritance
-
ColumnWriter
- Implements
- Derived
- Inherited Members
Properties
ColumnDescriptor
Get the ColumnDescriptor for the column.
public ColumnDescriptor ColumnDescriptor { get; }
Property Value
ColumnIndex
Get the index of the column within the row group.
public int ColumnIndex { get; }
Property Value
ElementType
Get the element Type of the data being written.
public abstract Type ElementType { get; }
Property Value
EstimatedBufferedStats
Estimated sizes of data buffered by this column writer (levels, values and dictionary) that has not yet been written to pages.
public BufferedWriterStats EstimatedBufferedStats { get; }
Property Value
Remarks
This does not include completed data pages held in memory while the column is dictionary encoded, which are reported by TotalCompressedBytes.
When the column is dictionary encoded, ValueBytes is a conservative estimate of the encoded dictionary indices size and may be non-zero when no values are buffered. After the column is closed, DictBytes continues to report the size of the dictionary, although the dictionary page has been written.
Handle
protected nint Handle { get; }
Property Value
LogicalTypeFactory
Get the LogicalTypeFactory for the Parquet file writer.
public LogicalTypeFactory LogicalTypeFactory { get; }
Property Value
LogicalWriteConverterFactory
Get the LogicalWriteConverterFactory for the Parquet file writer.
public LogicalWriteConverterFactory LogicalWriteConverterFactory { get; }
Property Value
RowWritten
Get the number of rows written to the column so far.
public long RowWritten { get; }
Property Value
TotalBytesWritten
Total uncompressed bytes of data and dictionary pages written by the page writer for this column so far.
public long TotalBytesWritten { get; }
Property Value
Remarks
This does not include completed data pages held in memory while the column is dictionary encoded, which are reported by TotalCompressedBytes.
TotalCompressedBytes
Total compressed bytes of data pages for this column that have been completed but not yet written by the page writer.
public long TotalCompressedBytes { get; }
Property Value
Remarks
Data pages are held in memory rather than written while the column is dictionary encoded, until the dictionary page is written when the column is closed. This is an estimate, as the size of each page header is over-estimated.
TotalCompressedBytesWritten
Total compressed bytes of data and dictionary pages written by the page writer for this column so far. If the column is uncompressed, this is equal to TotalBytesWritten.
public long TotalCompressedBytesWritten { get; }
Property Value
Remarks
This does not include completed data pages held in memory while the column is dictionary encoded, which are reported by TotalCompressedBytes.
Type
Get the physical type of the column.
public PhysicalType Type { get; }
Property Value
WriterProperties
Get the WriterProperties for the column.
public WriterProperties WriterProperties { get; }
Property Value
Methods
Apply<TReturn>(IColumnWriterVisitor<TReturn>)
Apply a visitor to the column writer.
public abstract TReturn Apply<TReturn>(IColumnWriterVisitor<TReturn> visitor)
Parameters
visitorIColumnWriterVisitor<TReturn>The visitor instance.
Returns
- TReturn
The result of the visitor operation.
Type Parameters
TReturnThe return type of the visitor.
Close()
public long Close()
Returns
Dispose()
Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
public void Dispose()
LogicalWriter(int)
Create a LogicalColumnWriter.
public LogicalColumnWriter LogicalWriter(int bufferLength = 4096)
Parameters
bufferLengthintThe buffer length in bytes. Default is 4KB.
Returns
- LogicalColumnWriter
A LogicalColumnWriter instance.
LogicalWriterOverride<TElement>(int)
Create a strongly-typed LogicalColumnWriter with an explicit element type override.
public LogicalColumnWriter<TElement> LogicalWriterOverride<TElement>(int bufferLength = 4096)
Parameters
bufferLengthintThe buffer length in bytes. Default is 4KB.
Returns
- LogicalColumnWriter<TElement>
A LogicalColumnWriter instance.
Type Parameters
TElementThe type of the data to write.
LogicalWriter<TElement>(int)
Create a strongly-typed LogicalColumnWriter without an explicit element type override.
public LogicalColumnWriter<TElement> LogicalWriter<TElement>(int bufferLength = 4096)
Parameters
bufferLengthintThe buffer length in bytes. Default is 4KB.
Returns
- LogicalColumnWriter<TElement>
A LogicalColumnWriter instance.
Type Parameters
TElementThe type of the data to write.