Marine Library  1.0
C++ library for Linux Networking Development
Classes | Namespaces | Typedefs | Functions
data_stream.hh File Reference

Data packing library using stream style interfaces. More...

#include <cassert>
#include <cstring>
#include <vector>
#include "impl/data_stream_impl.hh"

Go to the source code of this file.

Classes

class  CInByteStream
 Data unpacking interfaces. More...
 
class  COutByteStreamBasic< Data >
 Data packing interfaces. More...
 

Namespaces

 Manip
 Manipulators for stream interface APIs.
 

Typedefs

typedef COutByteStreamBasic< NS_IMPL::__buf_data< std::string > > COutByteStreamStr
 Use internal std::string as the underlying data buffer. More...
 
typedef COutByteStreamStr COutByteStream
 Same as COutByteStreamStr.
 
typedef COutByteStreamBasic< NS_IMPL::__buf_ref_data< std::string > > COutByteStreamStrRef
 Use external std::string as the underlying data buffer. More...
 
typedef COutByteStreamBasic< NS_IMPL::__buf_data< std::vector< char > > > COutByteStreamVec
 Use internal std::vector<char> as the underlying data buffer. More...
 
typedef COutByteStreamBasic< NS_IMPL::__buf_ref_data< std::vector< char > > > COutByteStreamVecRef
 Use external std::vector<char> as the underlying data buffer. More...
 
typedef COutByteStreamBasic< NS_IMPL::__buf_data< CCharBuffer< char > > > COutByteStreamBuf
 Use external char array as the underlying data buffer. More...
 

Functions

template<class T >
NS_IMPL::CManipulatorProtobuf< T > Manip::protobuf (T &msg, size_t size=size_t(-1))
 Pack/unpack Protocol Buffers messages. More...
 
template<typename T >
NS_IMPL::CManipulatorVarint< const T > Manip::varint (const T &val)
 Pack integer using Base 128 Varints encoding. More...
 
template<typename T >
NS_IMPL::CManipulatorVarint< T > Manip::varint (T &val)
 Unpack integer using Base 128 Varints encoding. More...
 
NS_IMPL::CManipulatorStubPush Manip::stub (size_t sz)
 Set up a stub. More...
 
NS_IMPL::CManipulatorStubPop Manip::stub_pop (bool align=false, bool check=false)
 Demolish the top most stub. More...
 
Mapip::raw

Pack/unpack a range of elements, without leading size field.

Mapip::raw is convenient for pack/unpack fixed number of elements, like an array, a string of characters, or a container.

Array
Sample code for CInByteStream:
CInByteStream in(buf, sz);
int c[5]; // an array to receive unpacked results
in >> Mapip::raw(c); // unpack 5 integers
// This is equivalent to:
// for(int i = 0;i < 5;++i)
// in >> c[i];
Sample code for COutByteStreamBasic:
COutByteStream out;
int c[5]; // an array to pack
out << Mapip::raw(c); // pack 5 integers
// This is equivalent to:
// for(int i = 0;i < 5;++i)
// out << c[i];
If size of array cannot be deduced automatically, a parameter must be provided.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
int * c = new int[sz]; // an array to receive unpacked results
in >> Mapip::raw(c, sz); // unpack 'sz' integers
// This is equivalent to:
// for(int i = 0;i < sz;++i)
// in >> c[i];
Sample code for COutByteStreamBasic:
COutByteStream out;
int * c = new int[sz]; // an array to pack
out << Mapip::raw(c, sz); // pack 'sz' integers
// This is equivalent to:
// for(int i = 0;i < sz;++i)
// out << c[i];
Containers
Most STL containers are well supported, e.g. vector, list, deque, set/multiset, map/multimap, etc.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
vector<int> c; // a container to receive unpacked results
in >> Mapip::raw(c, sz); // unpack 'sz' integers, and append them to 'c'
// This is equivalent to:
// for(int i = 0, v;i < sz;++i){
// in >> v;
// c.insert(c.end(), v);
// }
Sample code for COutByteStreamBasic:
COutByteStream out;
vector<int> c; // an vector to pack
out << Mapip::raw(c); // pack all elements in 'c'
// This is equivalent to:
// for(int i = 0;i < c.size();++i)
// out << c[i];
String
A C-style string is essentially an array of characters.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
char * c = new char[sz]; // an array of chars to receive unpacked results
in >> Mapip::raw(c, sz); // unpack 'sz' chars
// This is equivalent to:
// for(int i = 0;i < sz;++i)
// in >> c[i];
Sample code for COutByteStreamBasic:
COutByteStream out;
const char * c = "abcde"; // a C-style string to pack
out << Mapip::raw(c, strlen(c)); // pack a C-style string
// This is equivalent to:
// for(int i = 0;i < strlen(c);++i)
// out << c[i];

An std::string is a container of characters.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
string c; // an object to receive unpacked data
in >> Mapip::raw(c, len); // unpack 'len' bytes of data, and append them to 'c'
// This is equivalent to:
// for(int i = 0;i < len;++i){
// char v;
// in >> v;
// c.push_back(v);
// }
Sample code for COutByteStreamBasic:
COutByteStream out;
string c = "abcde"; // a string to pack
out << Mapip::raw(c); // pack the string
// This is equivalent to:
// for(int i = 0;i < c.size();++i)
// out << c[i];
Iterator range
Sometimes you only have a begin and an end iterators to do the packing/unpacking. A size_t * parameter might be provided to retrieve the number of elements actually packed/unpacked. If it doesn't matter, just ignore it.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
list<int> c;
list<int>::iterator first = c.begin(); // start iterator
list<int>::iterator last = c.end(); // end iterator
size_t sz;
in >> Mapip::raw(first, last, &sz); // unpack certain integers to range [first, last), and
// store the number of elements unpaced in 'sz'
// This is equivalent to:
// for(list<int>::iterator it = first;it != last;++it)
// in >> *it;
Sample code for COutByteStreamBasic:
COutByteStream out;
list<int> c;
list<int>::const_iterator first = c.begin(); // start iterator
list<int>::const_iterator last = c.end(); // end iterator
out << Mapip::raw(first, last); // pack certain integers from range [first, last), not
// interested in the number of elements packed
// This is equivalent to:
// for(list<int>::const_iterator it = first;it != last;++it)
// out << *it;
template<class T , size_t N>
NS_IMPL::CManipulatorRawPtr< T > Manip::raw (T(&c)[N])
 
template<class T >
NS_IMPL::CManipulatorRawPtr< T > Manip::raw (T *c, size_t sz)
 
template<class T >
NS_IMPL::CManipulatorRawPtr< T > Manip::raw (std::vector< T > &c, size_t sz)
 
template<class T >
NS_IMPL::CManipulatorRawPtr< const T > Manip::raw (const std::vector< T > &c, size_t *sz=NULL)
 
template<typename Char >
NS_IMPL::CManipulatorRawPtr< Char > Manip::raw (std::basic_string< Char > &c, size_t len)
 
template<typename Char >
NS_IMPL::CManipulatorRawPtr< const Char > Manip::raw (const std::basic_string< Char > &c, size_t *sz=NULL)
 
template<class ForwardIter >
NS_IMPL::CManipulatorRawRange< ForwardIter > Manip::raw (ForwardIter first, ForwardIter last, size_t *sz=NULL)
 
template<class T >
NS_IMPL::CManipulatorRawCont< T > Manip::raw (T &c, size_t sz)
 
template<class T >
NS_IMPL::CManipulatorRawCont< const T > Manip::raw (const T &c, size_t *sz=NULL)
 
Mapip::array

Pack/unpack a range of elements, with leading size field.

Mapip::array is convenient for pack/unpack a number of elements, like an array, a string of characters, or a container. It is very similar to Manip::raw, except that a size field (the number elements) is packed/unpacked first before the elements. By default, the type of leading size field is uint16_t, unless you specify it explicitly.

Array
Sample code for CInByteStream:
CInByteStream in(buf, sz);
int c[10]; // an array to receive unpacked results
uint8_t sz; // an integer to retrieve the number of elements actually unpacked
in >> Mapip::array(c, &sz); // firstly unpack an uint8_t to 'sz' as the number of following
// elements, then unpack 'sz' integers to 'c'
// This is equivalent to:
// in >> sz;
// for(int i = 0;i < sz;++i)
// in >> c[i];
Note that if sz is larger than the size c can hold, the whole operation fails with status of in set to non-zero.
Sample code for COutByteStreamBasic:
COutByteStream out;
int c[5]; // an array to pack
out << Mapip::array<uint16_t>(c); // pack a uint16_t equals to 5, then pack 5 integers
// This is equivalent to:
// out << uint16_t(5);
// for(int i = 0;i < 5;++i)
// out << c[i];
Note that you must specify the type of leading size.
If size of array cannot be deduced automatically, a parameter must be provided.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
int * c = new int[sz]; // an array to receive unpacked results
uint16_t real_sz; // an integer to retrieve the number of elements actually unpacked
in >> Mapip::array(c, sz, &real_sz);// firstly unpack a uin16_t to 'real_sz' as the number of
// following elements, then unpack 'real_sz' integers
// This is equivalent to:
// in >> real_sz;
// for(int i = 0;i < real_sz;++i)
// in >> c[i];
Note that if real_sz is larger than sz, the whole operation fails with status of in is set to non-zero.
Sample code for COutByteStreamBasic:
COutByteStream out;
int * c = new int[sz]; // an array to pack
out << Mapip::array<uint16_t>(c, sz); // pack a uint16_t equals to 'sz' first, then pack
// 'sz' integers
// This is equivalent to:
// out << uint16_t(sz);
// for(int i = 0;i < sz;++i)
// out << c[i];
Note that you must specify the type of leading size.
Containers
Most STL containers are well supported, e.g. vector, list, deque, set/multiset, map/multimap, etc.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
vector<int> c; // a container to receive unpacked results
in >> Mapip::array(c); // firstly unpack a uint16_t as the number of following elements,
// then unpack that number of integers, and append them to 'c'
// This is equivalent to:
// uint16_t sz;
// in >> sz;
// for(int i = 0, v;i < sz;++i){
// in >> v;
// c.insert(c.end(), v);
// }
If the leading size is not uint16_t, you can specify it like this:
in >> Mapip::array<uint8_t>(c); // firstly unpack a uint8_t as the number of following
// elements, then unpack that number of integers, and
// append them to 'c'
If there is a limitation of elements, an additional parameter may be provided, and the leading size has the same type as the second parameter:
uint32_t max_sz = 100;
in >> Mapip::array(c, max_sz); // firstly unpack a uint32_t as the number of following
// elements, then unpack that number of integers, and
// append them to 'c'
If the leading size unpacked is larger than max_sz, the operation fails and status of in is set to non-zero.
Sample code for COutByteStreamBasic:
COutByteStream out;
vector<int> c; // an vector to pack
out << Mapip::array(c); // firstly pack a uint16_t equals to 'c.size()', then pack all
// elements in 'c'
// This is equivalent to:
// out << uint16_t(c.size());
// for(int i = 0;i < c.size();++i)
// out << c[i];
Note that you can specify the type of leading size like this:
out << Mapip::array<uint8_t>(c); // firstly pack a uint8_t equals to 'c.size()', then pack
//all elements in 'c'
String
A C-style string is essentially an array of characters.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
char * c = new char[sz]; // an array of chars to receive unpacked results
uint16_t real_sz; // an integer to retrieve the number characters actually unpacked
in >> Mapip::array(c, sz, &real_sz); // firstly unpack a uint16_t to 'real_sz' as the
// number of following characters, then unpack
// 'real_sz' characters to 'c'
// This is equivalent to:
// in >> real_sz;
// for(int i = 0;i < real_sz;++i)
// in >> c[i];
If real_sz is larger than sz, this operation fails and status of in is set to non-zero.
Sample code for COutByteStreamBasic:
COutByteStream out;
const char * c = "abcde"; // a C-style string to pack
out << Mapip::array<uint8_t>(c, strlen(c)); // firstly pack a uint8_t equals to
// 'strlen(c)', then pack the content of 'c'
// This is equivalent to:
// out << uint8_t(strlen(c));
// for(int i = 0;i < strlen(c);++i)
// out << c[i];
Note that the type of leading size must be provided.
An std::string is a container of characters.
Sample code for CInByteStream:
CInByteStream in(buf, sz);
string c; // an object to receive unpacked data
in >> Mapip::array(c); // firstly unpack a uint16_t as the number of following characters,
// then unpack that number of bytes of data, and append them to 'c'
// This is equivalent to:
// uint16_t sz;
// in >> sz;
// for(int i = 0;i < sz;++i){
// char v;
// in >> v;
// c.push_back(v);
// }
You can specify the type of leading size like this:
in >> Mapip::array<uint8_t>(c); // firstly unpack a uint8_t as the number of following
// characters, then unpack that number of bytes of data,
// and append them to 'c'
If there is a limitation of bytes to unpack, an additional parameter may be provided, and the type of leading size is the same as the second parameter:
uint8_t max_sz = 100;
in >> Mapip::array(c, max_sz); // firstly unpack a uint8_t as the number of following
// characters, then unpack that number of bytes of data,
// and append them to 'c'
If the leading size unpacked is larger than max_sz, the operation fails and status of in is set to non-zero.
Sample code for COutByteStreamBasic:
COutByteStream out;
string c = "abcde"; // a string to pack
out << Mapip::array(c); // firstly pack a uint16_t equals to 'c.size()', then pack the
// content of 'c'
// This is equivalent to:
// out << uint16_t(c.size());
// for(int i = 0;i < c.size();++i)
// out << c[i];
If the type of leading size isn't uint16_t, you can specify it like this:
out << Mapip::array<uint8_t>(c); // firstly pack a uint8_t equals to 'c.size()', then pack
// the content of 'c'
Iterator range (pack only)
Sometimes you only have a begin and an end iterators to do the packing.
Sample code for COutByteStreamBasic:
COutByteStream out;
list<int> c;
list<int>::const_iterator first = c.begin(); // start iterator
list<int>::const_iterator last = c.end(); // end iterator
out << Mapip::array<uint16_t>(first, last);// firstly pack a uint16_t equals to the number of
// elements in range [first, last), then pack all
// integers the range
// This is equivalent to:
// out << uint16_t(distance(first, last));
// for(list<int>::const_iterator it = first;it != last;++it)
// out << *it;
An additional integer parameter could be provided to retrieve the number of elements packed, and it will have the same type of the leading size:
uint8_t sz;
out << Mapip::array(first, last, &sz); // firstly pack a uint8_t equals to the number of
// elements in range [first, last), then pack all
// integers in the range
template<typename LenT , class T , size_t N>
NS_IMPL::CManipulatorArrayPtr< LenT, T > Manip::array (T(&c)[N], LenT *real_sz)
 
template<typename LenT , class T , size_t N>
NS_IMPL::CManipulatorArrayPtr< LenT, const T > Manip::array (const T(&c)[N])
 
template<typename LenT , class T >
NS_IMPL::CManipulatorArrayPtr< LenT, T > Manip::array (T *c, size_t sz, LenT *real_sz)
 
template<typename LenT , class T >
NS_IMPL::CManipulatorArrayPtr< LenT, const T > Manip::array (const T *c, size_t sz)
 
template<typename LenT , class T >
NS_IMPL::CManipulatorArrayCont< LenT, T > Manip::array (T &c, LenT max_size=0)
 
template<typename LenT , class T >
NS_IMPL::CManipulatorArrayCont< LenT, const T > Manip::array (const T &c)
 
template<class T >
NS_IMPL::CManipulatorArrayCont< uint16_t, T > Manip::array (T &c, uint16_t max_size=0)
 
template<class T >
NS_IMPL::CManipulatorArrayCont< uint16_t, const T > Manip::array (const T &c)
 
template<typename LenT , class ForwardIter >
NS_IMPL::CManipulatorArrayRange< LenT, ForwardIter > Manip::array (ForwardIter first, ForwardIter last, LenT *sz=NULL)
 
Byte order

APIs for changing CInByteStream / COutByteStreamBasic object's byte order, both permanently and temporarily.

Change byte order permanently.
Sample code:
CInByteStream in(buf, sz);
COutByteStream out;
// Set the underlying data buffer as Net Byte Order (Big Endian).
in >> Mapip::net_order;
out << Manip::net_order;
// Set the underlying data buffer as Host Byte Order.
in >> Mapip::host_order;
out << Manip::host_order;
// Set the underlying data buffer as Little Endian.
in >> Mapip::little_endian;
out << Manip::little_endian;
// Set the underlying data buffer as Big Endian (Net Byte Order).
in >> Mapip::big_endian;
out << Manip::big_endian;
Change byte order temporarily.
Set byte order and pack/unpack data, then restore old byte order.
Sample code:
CInByteStream in(buf, sz);
COutByteStream out;
int val = 10;
in >> Mapip::net_order_value(val); // unpack an integer using Net Byte Order, despite
// the actual byte order of 'in'
out << Mapip::net_order_value(val); // pack an integer using Net Byte Order, dspite the
// actual byte order of 'out'
in >> Mapip::host_order_value(val); // unpack an integer using Host Byte Order, despite
// the actual byte order of 'in'
out << Mapip::host_order_value(val); // pack an integer using Host Byte Order, dspite the
// actual byte order of 'out'
in >> Mapip::little_endian_value(val); // unpack an integer using Little Endian, despite
// the actual byte order of 'in'
out << Mapip::little_endian_value(val); // pack an integer using Little Endian, dspite the
// actual byte order of 'out'
in >> Mapip::big_endian_value(val); // unpack an integer using Big Endian, despite the
// actual byte order of 'in'
out << Mapip::big_endian_value(val); // pack an integer using Big Endian, dspite the actual
// byte order of 'out'
void Manip::net_order (NS_IMPL::CDataStreamBase &ds)
 
void Manip::host_order (NS_IMPL::CDataStreamBase &ds)
 
void Manip::little_endian (NS_IMPL::CDataStreamBase &ds)
 
void Manip::big_endian (NS_IMPL::CDataStreamBase &ds)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< T > Manip::net_order_value (T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< const T > Manip::net_order_value (const T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< T > Manip::host_order_value (T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< const T > Manip::host_order_value (const T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< T > Manip::little_endian_value (T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< const T > Manip::little_endian_value (const T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< T > Manip::big_endian_value (T &val)
 
template<class T >
NS_IMPL::CManipulatorValueByteOrder< const T > Manip::big_endian_value (const T &val)
 
Current position

APIs for changing current position, both permanently and temporarily.

Change current postion permanently
Sample code:
CInByteStream in(buf, sz);
COutByteStream out;
// Set current position absolutely
in >> Mapip::seek(10); // Same as in.seek(10)
out << Mapip::seek(10); // Same as out.seek(10)
// Move current position relatively
in >> Mapip::skip(10); // Same as in.skip(10)
out << Mapip::skip(10); // Same as out.skip(10)
uint32_t off = -10;
in >> Mapip::skip(&off); // Same as in.skip(off)
out << Mapip::skip(&off); // Same as out.skip(off)
// Fill while moving
int fill = 'a';
in >> Mapip::skip(10, fill); // Same as in.skip(10, fill)
out << Mapip::skip(10, fill); // Same as out.skip(10, fill)
uint32_t off = -10;
in >> Mapip::skip(&off, fill); // Same as in.skip(off, fill)
out << Mapip::skip(&off, fill); // Same as out.skip(off, fill)
Change current postion temporarily
Move current position and pack/unpack data, then restore old current position.
Sample code:
CInByteStream in(buf, sz);
COutByteStream out;
int val = 100;
// Unpack data from an absolute position
in >> Mapip::offset_value(10, val);
// This is equivalent to:
// size_t old = in.cur();
// in.seek(10);
// in >> val;
// in.seek(old);
// Pack data to an absolute position
out << Mapip::offset_value(10, val);
// This is equivalent to:
// size_t old = out.cur();
// out.seek(10);
// out << val;
// out.seek(old);
Insert data (pack only)
Sometimes you need to insert a value to a particular position in already packed data, which means all existing data after that position need to move forward, to make room for the value. This operation could be complex without Manip::insert.
Sample code:
COutByteStream out;
int val = 100;
out << Mapip::insert(10, val); // pack 'val' at absolute position 10. Any existing data
// after this position will move forward by 'sizeof val'
// bytes to make room for 'val'
NS_IMPL::CManipulatorSeek Manip::seek (size_t pos)
 
NS_IMPL::CManipulatorSkip Manip::skip (ssize_t off)
 
NS_IMPL::CManipulatorSkipFill Manip::skip (ssize_t off, int fill)
 
template<typename T >
NS_IMPL::CManipulatorSkipPtr< T > Manip::skip (T *off)
 
template<typename T >
NS_IMPL::CManipulatorSkipPtrFill< T > Manip::skip (T *off, int fill)
 
template<class T >
NS_IMPL::CManipulatorOffsetValue< T > Manip::offset_value (size_t pos, T &val)
 
template<class T >
NS_IMPL::CManipulatorOffsetValue< const T > Manip::offset_value (size_t pos, const T &val)
 
template<class T >
NS_IMPL::CManipulatorInsert< T > Manip::insert (size_t pos, const T &val)
 
Manip::end

End packing/unpacking operations.

When all packing/unpacking operations finish, it's a good practice to announce an end explicitly.
For unpacking operations (CInByteStream), left data size will be checked. If it's not zero, status of CInByteStream object will be set to non-zero. Because usually some left data means you have missed something, or there is a misunderstanding about the protocol.
For packing operations (COutByteStreamBasic), it's always necessary to end. Because the underlying data buffer (whether it's internal or external) may have reserved some room for performance, so an adjustment is critical to correct its size. Besides, it will call corresponding COutByteStreamBasic::finish to export data.

Note
Ending operation will clear all stubs, without any alignments or checks.
See also
COutByteStreamBasic::finish
NS_IMPL::CManipulatorEnd< void, void > Manip::end ()
 End operations for CInByteStream, COutByteStreamStrRef, COutByteStreamVecRef. More...
 
template<typename SizeT >
NS_IMPL::CManipulatorEnd< SizeT *, void > Manip::end (SizeT *sz)
 End operations for COutByteStreamBuf, COutByteStreamStrRef, COutByteStreamVecRef. More...
 
template<class BufT >
NS_IMPL::CManipulatorEnd< BufT, void > Manip::end (BufT &buf)
 End operations for COutByteStream / COutByteStreamStr, COutByteStreamStrRef, COutByteStreamVec, COutByteStreamVecRef. More...
 
template<typename CharT >
NS_IMPL::CManipulatorEnd< CharT *, size_t * > Manip::end (CharT *buf, size_t *sz)
 End operations for COutByteStream / COutByteStreamStr, COutByteStreamStrRef, COutByteStreamVec, COutByteStreamVecRef, COutByteStreamBuf. More...
 

Detailed Description

Data packing library using stream style interfaces.

Author
Zhao DAI

Typedef Documentation

typedef COutByteStreamBasic<NS_IMPL::__buf_data<CCharBuffer<char> > > COutByteStreamBuf

Use external char array as the underlying data buffer.

typedef COutByteStreamBasic<NS_IMPL::__buf_data<std::string> > COutByteStreamStr

Use internal std::string as the underlying data buffer.

typedef COutByteStreamBasic<NS_IMPL::__buf_ref_data<std::string> > COutByteStreamStrRef

Use external std::string as the underlying data buffer.

typedef COutByteStreamBasic<NS_IMPL::__buf_data<std::vector<char> > > COutByteStreamVec

Use internal std::vector<char> as the underlying data buffer.

typedef COutByteStreamBasic<NS_IMPL::__buf_ref_data<std::vector<char> > > COutByteStreamVecRef

Use external std::vector<char> as the underlying data buffer.

Generated by   doxygen 1.8.11