|| || || || || ||

---

Impala::Util::IOBuffer Class Reference

Base class for I/O buffers. More...

#include <IOBuffer.h>

Inheritance diagram for Impala::Util::IOBuffer:

[legend]Collaboration diagram for Impala::Util::IOBuffer:

[legend]List of all members.

Public Types
typedef Int64	PositionType
Public Member Functions
	IOBuffer ()
	create an empty buffer where you can write in
	IOBuffer (PositionType length)
	create a buffer of given size
	IOBuffer (PositionType length, unsigned char *buffer)
	after this call the IOBuffer owns the memory pointed to by buffer, no copy is made.
	IOBuffer (String s)
	This c'tor copies the contents of the string.
virtual	~IOBuffer ()
virtual bool	Valid ()
virtual PositionType	Available ()
	Number of bytes til end of controlled entity.
virtual int	AvailableInt32 ()
virtual PositionType	Size ()
	total number of bytes in controlled entity
virtual void	SetSize (PositionType size)
template<class BackInsertIterator>
void	ReadStrings (BackInsertIterator bi, bool skipComments, int count)
template<class Iterator>
bool	WriteStrings (Iterator begin, Iterator end, int count=-1)
unsigned char *	GetBuffer ()
	this function reads everything to enable MPI_Broadcast and returns the buffer
void	SetBuffer (unsigned char *buf, PositionType size)
	after this call the IOBuffer owns the memory pointed to by buffer, no copy is made Sander and I (Michiel) already talked about how this should probably be only a constructor but I need it now, and we'll redisign later ;-)
void	SetUseChannel (Channel *channel, String channelFile, String tmpFile)
	Indicate that data should be sent across given channel on destruction.
I/O
These functions read from the memory buffer, not necessarily the controlled entity
virtual Int64	Read (void *buf, Int64 bytesToRead)
	returns number of bytes read
Int64	ReadBzip (void *buf, Int64 bytesToRead)
Int64	ReadZlib (void *buf, Int64 bytesToRead)
virtual String	ReadLine ()
	only for ascii (not binary) entities
virtual Int64	Gets (char *buf, Int64 bytesToRead)
virtual void	NativeTypeRead (Int8 *ptr)
virtual void	NativeTypeRead (UInt8 *ptr)
virtual void	NativeTypeRead (Int16 *ptr)
virtual void	NativeTypeRead (UInt16 *ptr)
virtual void	NativeTypeRead (Int32 *ptr)
virtual void	NativeTypeRead (UInt32 *ptr)
virtual void	NativeTypeRead (Int64 *ptr)
virtual void	NativeTypeRead (UInt64 *ptr)
virtual void	NativeTypeRead (Real32 *ptr)
virtual void	NativeTypeRead (Real64 *ptr)
virtual Int64	Write (const void *buf, Int64 bytesToWrite)
void	WriteBzip (void *srcBuf, Int64 bytesToWrite)
void	WriteZlib (void *srcBuf, Int64 bytesToWrite)
virtual Int64	Puts (const char *buf)
virtual Int64	Puts (const String &str)
virtual void	NativeTypeWrite (Int8 val)
virtual void	NativeTypeWrite (UInt8 val)
virtual void	NativeTypeWrite (Int16 val)
virtual void	NativeTypeWrite (UInt16 val)
virtual void	NativeTypeWrite (Int32 val)
virtual void	NativeTypeWrite (UInt32 val)
virtual void	NativeTypeWrite (Int64 val)
virtual void	NativeTypeWrite (UInt64 val)
virtual void	NativeTypeWrite (Real32 val)
virtual void	NativeTypeWrite (Real64 val)
Positions
The hiding of positions enables us to implement complex behaviour like caching or asynchronous writing
virtual void	SetPosition (PositionType position)
virtual void	SetPositionAndSize (PositionType position)
virtual PositionType	GetPosition ()
virtual void	Rewind ()
virtual void	FastForward ()
	strange name perhaps? this function goes to the end of the controlled entity
virtual PositionType	Seek (PositionType offset, int whence)
Static Public Attributes
static const UInt32	cBzipBlockSize = 16 * 1024 * 1024
Protected Member Functions
template<class NativeType>
void	DoNativeTypeRead (NativeType *ptr)
template<class NativeType>
void	DoNativeTypeWrite (NativeType val)
bool	WriteIOBufferToChannel ()
void	CheckDataChannelWrite ()
Private Attributes
unsigned char *	mBuffer
PositionType	mSize
PositionType	mPosition
Channel *	mDataChannel
String	mDataChannelFile
String	mDataChannelTmp
	ILOG_VAR_DEC

---

Detailed Description

Base class for I/O buffers.

Derived classes interact with file, socket, DB blob, etc.

terminology:

• this is the buffer
• the file, socket, DB, whatever is the controlled entity

purpose:

• to read and write collections of unsigned bytes: there is other code to do parsing and conversions (iostreams to the rescue! :D)
• for large files we want to prevent transferring the whole controlled entity to memory
• for mpi we want to read the whole controlled entity to memory to MPI_Broadcast it

notes:

• not all controlled entities may support random access (streams), perhaps these cache everything, so that we can have random access in the part that has arrived and random access in parts that have not arrived can be given by waiting for the stream to yield that information or in case of write to have asynchronous writing
• which functions to make virtual and which not remains to be decided

Definition at line 52 of file IOBuffer.h.

---

The documentation for this class was generated from the following file:

• Util/IOBuffer.h

---