AMOS::Bank_t

AMOS::Bank_t Class Reference

An AMOS data bank for efficiently storing Bankable data types. More...

#include <Bank_AMOS.hh>

Inheritance diagram for AMOS::Bank_t:

[legend]
Collaboration diagram for AMOS::Bank_t:
[legend]
List of all members.

Public Types

typedef int64_t bankstreamoff
 64-bit stream offset for largefiles

Public Member Functions

 Bank_t (NCode_t type)
 Constructs an empty Bank_t of objects with a certain NCode.
 Bank_t (const std::string &type)
 ~Bank_t ()
 Destroys a Bank_t object.
void append (IBankable_t &obj)
 Appends a Bankable object to the bank.
void assignEID (ID_t iid, const std::string &eid)
 Assigns an EID to an existing object.
void assignIID (const std::string &eid, ID_t iid)
 Assigns an object a new IID.
void clean ()
 Reorganizes the bank and removes all residual deleted objects.
void clear ()
 Clears a bank by erasing all it's objects and its ID map.
void close ()
 Closes a bank on disk.
void concat (Bank_t &source)
 Concatenates another bank to the end of this bank.
void create (const std::string &dir, BankMode_t mode=B_READ|B_WRITE)
 Forcibly creates and opens a bank on disk.
void destroy ()
 Closes and removes a bank from disk.
bool empty () const
 Returns true if bank is empty.
bool exists (const std::string &dir) const
 Checks for the existence of a another bank with similar type.
bool existsEID (const std::string &eid) const
 Returns true if EID exists in the bank.
bool existsIID (ID_t iid) const
 Returns true if IID exists in the bank.
void fetch (ID_t iid, IBankable_t &obj)
 Fetches a Bankable object from the bank by its IID.
void fetch (const std::string &eid, IBankable_t &obj)
 Fetches a Bankable object from the bank by its EID.
void fetchFix (ID_t iid, IBankable_t &obj)
 Fetches the fixed length part of a Bankable object by its IID.
void fetchFix (const std::string &eid, IBankable_t &obj)
 Fetches the fixed length part of a Bankable object by its EID.
const IDMap_t & getIDMap () const
 Get the current (IID <-> EID) -> BID map for the bank.
Size_t getIDMapSize () const
 Get the number of objects in the Bank with and IID or EID.
Size_t getIndexSize () const
 Get the number of indices used by the bank.
ID_t getMaxIID () const
 Searches through the bank for the largest IID.
ID_t getMaxBID () const
 Searches through the bank for the largest BID.
Size_t getSize () const
 Get the size of the bank, i.e. the number of stored records.
signed char getStatus () const
 Get the bank status.
NCode_t getType () const
 Get the unique bank type identifier.
bool isOpen () const
 Check the bank's open status.
ID_t lookupBID (ID_t iid) const
 Converts an IID to a BID, throw exception on failure.
ID_t lookupBID (const std::string &eid) const
 Converts an EID to a BID, throw exception on failure.
const std::string & lookupEID (ID_t iid) const
 Converts an IID to an EID.
ID_t lookupIID (const std::string &eid) const
 Converts an EID to an IID.
void open (const std::string &dir, BankMode_t mode=B_READ|B_WRITE)
 Opens a bank on disk.
void remove (ID_t iid)
 Removes an object from the bank by its IID.
void remove (const std::string &eid)
 Removes an object from the bank by its EID.
void replace (ID_t iid, IBankable_t &obj)
 Replaces an object in the bank by its IID.
void replace (const std::string &eid, IBankable_t &obj)
 Replaces an object in the bank by its EID.
void setStatus (signed char status)
 Set the bank status.

Static Public Attributes

static const std::string BANK_VERSION = "2.8"
 current bank version
static const std::string IFO_STORE_SUFFIX = ".ifo"
 the informational store
static const std::string MAP_STORE_SUFFIX = ".map"
 the ID map store
static const std::string LCK_STORE_SUFFIX = ".lck"
 the ifo store file lock
static const std::string FIX_STORE_SUFFIX = ".fix"
 the fixed length stores
static const std::string VAR_STORE_SUFFIX = ".var"
 the variable length stores
static const std::string TMP_STORE_SUFFIX = ".tmp"
 the temporary store
static const char WRITE_LOCK_CHAR = 'w'
 write lock char
static const char READ_LOCK_CHAR = 'r'
 read lock char

Protected Types

 I_OPEN
 I_CREATE
 I_CLOSE
enum  IFOMode_t { I_OPEN, I_CREATE, I_CLOSE }

Protected Member Functions

void addPartition (bool create)
 Adds a new partition to the partition list.
void appendBID (IBankable_t &obj)
 Append an object, thus assigning it the last BID.
void fetchBID (ID_t bid, IBankable_t &obj)
 Fetch an object by BID.
void fetchBIDFix (ID_t iid, IBankable_t &obj)
 Fetch the fixed record for an object by BID.
BankPartition_t * getPartition (ID_t id)
 Returns the requested BankPartition, opening it if necessary.
BankPartition_t * getLastPartition ()
 Same as getPartition, but returns the current final partition.
void init ()
 Initializes bank variables.
BankPartition_t * localizeBID (ID_t &bid)
 Gets the partition and local identifier.
void lockIFO ()
 Obtains a file lock on the info store of the current bank.
BankPartition_t * openPartition (ID_t id)
 Do not use this function, use getPartition instead.
void removeBID (ID_t bid)
 Remove an object by BID.
void replaceBID (ID_t bid, IBankable_t &obj)
 Replace an object by BID.
void setMode (BankMode_t mode)
 validates and sets the BankMode
void syncIFO (IFOMode_t mode)
 Syncs the IFO store with in-memory data.
void touchFile (const std::string &path, int mode, bool create)
 Opens or creates a file, throwing exception on failure.
void unlockIFO ()
 Releases the file lock on the info store of the current bank.
 Bank_t (const Bank_t &source)
 Copy constructor.
Bank_t & operator= (const Bank_t &source)
 Assignment (copy) operator.

Protected Attributes

NCode_t banktype_m
 the type of objects stored in this bank
Size_t buffer_size_m
 size of the I/O buffer
Size_t max_partitions_m
 maximum number of open partitions
bool is_open_m
 open status of the bank
signed char status_m
 bank status
BankMode_t mode_m
 mode of the bank, B_READ|B_WRITE|B_SPY
std::string store_dir_m
 the disk store directory
std::string store_pfx_m
 the disk store prefix (including dir)
Size_t fix_size_m
 size of entire fixed length record
Size_t partition_size_m
 records per disk store partition
ID_t last_bid_m
 the last bank bid (1 based)
ID_t max_bid_m
 maximum bid given the current partitioning
ID_t nbids_m
 number of non-deleted bids
Size_t npartitions_m
 number of partitions
std::deque< BankPartition_t * > opened_m
 opened partitions
std::vector< BankPartition_t * > partitions_m
 all partitions
IDMap_t idmap_m
 the IDMap IID <-> EID to BID

Static Protected Attributes

static const Size_t DEFAULT_BUFFER_SIZE = 1024
 IO buffer size.
static const Size_t DEFAULT_PARTITION_SIZE = 1000000
 records per partition
static const Size_t MAX_OPEN_PARTITIONS = 20
 open partitions

Classes

class  BankPartition_t
 A single partition of the file-based bank. More...

Detailed Description

An AMOS data bank for efficiently storing Bankable data types.

Efficiently handles large numbers of Bankable data types and stores them to disk for future retrieval. The bank will upkeep a hash map of IIDs, EIDs and BIDs. All bank append/fetch style operations can be performed on either IIDs or EIDs.

If data not included in a Bankable type needs to be stored in a bank, please derive a new Bankable type and implement the appropriate virtual functions to allow the type to be stored in a bank. Developers should note that the IID, EID, and BIDs are never stored in the data partition, rather they are stored as an IDMap in a seperate partition.

A quick review of the identifiers. A BID is a bank identifier used only by the bank for indexing its contents, and is only useful for debugging purposes. The BID ordering will be the order in which the objects were appended (1 based). An IID is the universal AMOS ID that is used for all object links and most operations (append/fetch). An EID is a external ID string of arbitrary length, used less often due to the efficiency sacrifice of dealing with strings (EID) over ints (IID).

Definition at line 333 of file Bank_AMOS.hh.

Member Typedef Documentation

typedef int64_t AMOS::Bank_t::bankstreamoff

64-bit stream offset for largefiles

Definition at line 617 of file Bank_AMOS.hh.

Member Enumeration Documentation

enum AMOS::Bank_t::IFOMode_t [protected]

Enumerator:
I_OPEN 
I_CREATE 
I_CLOSE 

Definition at line 342 of file Bank_AMOS.hh.

Constructor & Destructor Documentation

AMOS::Bank_t::Bank_t ( const Bank_t &  source  )  [protected]

Copy constructor.

Private as to disallow copying

AMOS::Bank_t::Bank_t ( NCode_t  type  )  [inline]

Constructs an empty Bank_t of objects with a certain NCode.

Initializes members and sets bank type to the supplied value. All future operations on this bank must be made with a Bankable type that is compatibile with the supplied NCode.

Once a bank is created with a certain NCode, only objects compatible with that NCode can be used with that bank. For instance, if a bank is constructed with 'Bank_t mybank (Read_t::NCODE);', only Read_t objects could be used with mybank. Also, if a static NCode member is not available 'Bank_t mybank ("RED");' will also work.

Parameters:
type The type of bank to construct

Definition at line 649 of file Bank_AMOS.hh.

References buffer_size_m, DEFAULT_BUFFER_SIZE, init(), MAX_OPEN_PARTITIONS, max_partitions_m, and status_m.

AMOS::Bank_t::Bank_t ( const std::string &  type  )  [inline]

Definition at line 660 of file Bank_AMOS.hh.

References buffer_size_m, DEFAULT_BUFFER_SIZE, init(), MAX_OPEN_PARTITIONS, max_partitions_m, and status_m.

AMOS::Bank_t::~Bank_t (  )  [inline]

Destroys a Bank_t object.

Frees all occupied memory, flushes all buffers and closes any open files.

Definition at line 675 of file Bank_AMOS.hh.

References close(), and is_open_m.

Member Function Documentation

void Bank_t::addPartition ( bool  create  )  [protected]

Adds a new partition to the partition list.

Simply appends a new partition to the list, and leaves it unopened. Will throw an exception if unable to create/open partition. If the create flag is set, new files will be created and truncated to zero, otherwise its contents will be left undisturbed.

Parameters:
create Create new file or truncate existing
Precondition:
There are adequate permissions in the bank directory
Postcondition:
npartitions_m and max_iid_m reflect new partitioning
Exceptions:
IOException_t 
Returns:
void

Definition at line 60 of file Bank_AMOS.cc.

References buffer_size_m, DEFAULT_PARTITION_SIZE, FILE_MODE, AMOS::Bank_t::BankPartition_t::fix_name, FIX_STORE_SUFFIX, max_bid_m, npartitions_m, AMOS::NULL_STRING, partition_size_m, partitions_m, store_pfx_m, touchFile(), AMOS::Bank_t::BankPartition_t::var_name, and VAR_STORE_SUFFIX.

Referenced by appendBID(), clean(), concat(), AMOS::BankStream_t::operator<<(), and syncIFO().

void Bank_t::append ( IBankable_t &  obj  ) 

Appends a Bankable object to the bank.

Appends a Bankable object to the bank. The modified and removed flags of the object are cleared, because the object is newly appended. If the object has a non-empty IID/EID value, the bank will include the IID/EID in the IDMap to allow for future IID <-> EID conversions. Note that an object can be appended with a NULL IID and EID, but this will make it accessible only by iteration via a BankStream_t.

Parameters:
obj The Bankable object to append
Precondition:
The bank is open for writing

obj is compatible with the current NCode bank type

There is no IID/EID of this object already in the bank

Postcondition:
obj's modified and removed flags are cleared

obj IID/EID and assigned BID are added to the IDMap

Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 97 of file Bank_AMOS.cc.

References appendBID(), AMOS::IBankable_t::eid_m, idmap_m, AMOS::IBankable_t::iid_m, AMOS::IDMap_t::insert(), last_bid_m, and AMOS::IDMap_t::remove().

void Bank_t::appendBID ( IBankable_t &  obj  )  [protected]

Append an object, thus assigning it the last BID.

Definition at line 114 of file Bank_AMOS.cc.

References addPartition(), AMOS_THROW_ARGUMENT, AMOS_THROW_IO, AMOS::B_WRITE, banktype_m, AMOS::Bank_t::BankPartition_t::fix, fix_size_m, AMOS::IBankable_t::flags_m, getLastPartition(), AMOS::IBankable_t::getNCode(), AMOS::BankFlags_t::is_modified, is_open_m, AMOS::BankFlags_t::is_removed, last_bid_m, max_bid_m, mode_m, nbids_m, AMOS::Bank_t::BankPartition_t::var, AMOS::writeLE(), and AMOS::IBankable_t::writeRecord().

Referenced by append().

void AMOS::Bank_t::assignEID ( ID_t  iid,
const std::string &  eid 
)

Assigns an EID to an existing object.

Assigns an EID to an object that currently has only an IID. Only affects the IDMap and is therefore much more efficient than the replace method. Will throw an exception if the IID does not exist or if the new EID already exists.

Parameters:
iid The IID of the existing object
eid The EID to assign to the object
Precondition:
The bank is open for read/writing

The specified IID exists in the bank

The new EID does not exist in the bank

Exceptions:
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

void AMOS::Bank_t::assignIID ( const std::string &  eid,
ID_t  iid 
)

Assigns an object a new IID.

Assigns an IID to an object that currently has only an EID. Only affects the IDMap and is therefore much more efficient than the replace method. Will throw an exception if the IID does not exist or if the new EID already exists.

Parameters:
eid The EID of the existing object
iid The IID to assign to the object
Precondition:
The bank is open for read/writing

The specified EID exists in the bank

The new IID does not exist in the bank

Exceptions:
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

void Bank_t::clean (  ) 

Reorganizes the bank and removes all residual deleted objects.

Removes all objects waiting for deletion from disk. Also cleans up rubbish data left over from past replace operations. This is a costly operation, as it requires the entire bank be copied to a temporary store.

Precondition:
The bank is open for read/writing
Exceptions:
IOException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 194 of file Bank_AMOS.cc.

References addPartition(), AMOS_THROW_IO, AMOS::B_READ, AMOS::B_WRITE, banktype_m, clear(), concat(), create(), destroy(), DIR_MODE, fix_size_m, idmap_m, is_open_m, isOpen(), last_bid_m, mode_m, nbids_m, npartitions_m, partition_size_m, partitions_m, store_pfx_m, and TMP_STORE_SUFFIX.

Referenced by AMOS::BankStream_t::clean().

void Bank_t::clear (  ) 

Clears a bank by erasing all it's objects and its ID map.

Erases all the objects in a bank, but keeps the bank open at the current location. Only throws an exception if the bank is not open for writing. Won't complain if some of the partitions won't unlink etc. Has no effect on a closed bank.

Precondition:
The bank is open for writing
Exceptions:
IOException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 244 of file Bank_AMOS.cc.

References AMOS_THROW_IO, AMOS::B_WRITE, banktype_m, AMOS::IDMap_t::clear(), idmap_m, is_open_m, last_bid_m, max_bid_m, mode_m, nbids_m, npartitions_m, AMOS::NULL_ID, opened_m, partition_size_m, partitions_m, and AMOS::IDMap_t::setType().

Referenced by clean(), AMOS::BankStream_t::clear(), and destroy().

void Bank_t::close (  ) 

Closes a bank on disk.

Flushes all files, closes all files and re-initializes members. Has no effect on a closed bank.

Exceptions:
IOException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 273 of file Bank_AMOS.cc.

References AMOS_THROW_IO, AMOS::B_WRITE, I_CLOSE, idmap_m, init(), is_open_m, MAP_STORE_SUFFIX, mode_m, npartitions_m, partitions_m, store_pfx_m, syncIFO(), and AMOS::IDMap_t::write().

Referenced by AMOS::BankStream_t::close(), and ~Bank_t().

void Bank_t::concat ( Bank_t &  source  ) 

Concatenates another bank to the end of this bank.

Conceptually performs an append operation on every object in the source bank, but with more efficiency. As a side effect, incoming source records are cleaned (see Bank_t::clean() method), therefore removing all objects waiting for deletion. The two banks must have entirely disjoint IDMaps.

Note:
The source bank is unchanged by this operation, however it cannot be made a const because of I/O open/close operations.
Parameters:
source The bank to concat to the end of the current bank
Precondition:
The bank is open for writing

The source bank is open for reading

source is compatible with the current NCode bank type

The current and source IDMaps are disjoint

Postcondition:
The source bank is unaffected
Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Definition at line 305 of file Bank_AMOS.cc.

References addPartition(), AMOS_THROW_ARGUMENT, AMOS_THROW_IO, AMOS::B_READ, AMOS::B_WRITE, banktype_m, AMOS::IDMap_t::begin(), AMOS::IDMap_t::HashTriple_t::eid, AMOS::IDMap_t::end(), AMOS::Bank_t::BankPartition_t::fix, fix_size_m, getIDMap(), getLastPartition(), getPartition(), idmap_m, AMOS::IDMap_t::HashTriple_t::iid, AMOS::IDMap_t::insert(), is_open_m, AMOS::BankFlags_t::is_removed, last_bid_m, max_bid_m, mode_m, nbids_m, npartitions_m, AMOS::readLE(), AMOS::IDMap_t::remove(), AMOS::SafeMalloc(), AMOS::SafeRealloc(), AMOS::Bank_t::BankPartition_t::var, and AMOS::writeLE().

Referenced by clean(), and AMOS::BankStream_t::concat().

void AMOS::Bank_t::create ( const std::string &  dir,
BankMode_t  mode = B_READ|B_WRITE 
)

Forcibly creates and opens a bank on disk.

Forcibly creates and opens a bank. Any pre-existing bank of this type will be destroyed and any conflicting files will be overwritten if permissions allow. If you wish not to overwrite an existing bank, use the exists method to first check for a conflicting bank of this type. An open bank will first be closed before the new one is created.

Parameters:
dir The directory in which to create the bank
mode The mode of the bank (B_READ | B_WRITE)
Precondition:
mode includes B_WRITE

sufficient read/write/exe permissions for dir and bank files

Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Referenced by clean(), and AMOS::BankStream_t::create().

void Bank_t::destroy (  ) 

Closes and removes a bank from disk.

Closes the bank and unlinks all files and empty directories. Only throws an exception if the bank is not open for writing. Won't complain if some of the partitions won't unlink etc.

Precondition:
The bank is open for writing
Exceptions:
IOException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 468 of file Bank_AMOS.cc.

References AMOS_THROW_IO, clear(), IFO_STORE_SUFFIX, init(), is_open_m, LCK_STORE_SUFFIX, MAP_STORE_SUFFIX, mode_m, store_dir_m, and store_pfx_m.

Referenced by clean(), and AMOS::BankStream_t::destroy().

bool AMOS::Bank_t::empty (  )  const [inline]

Returns true if bank is empty.

Definition at line 845 of file Bank_AMOS.hh.

References getSize().

bool AMOS::Bank_t::exists ( const std::string &  dir  )  const

Checks for the existence of a another bank with similar type.

Only checks for the existence and readability of the info store, does not perform any validity checking on the other partitions. Can be called while a bank is open or closed.

Parameters:
dir The directory in which to look
Returns:
true if IFO store exists with read permissions, else false

Referenced by existsEID(), and existsIID().

bool AMOS::Bank_t::existsEID ( const std::string &  eid  )  const [inline]

Returns true if EID exists in the bank.

Definition at line 867 of file Bank_AMOS.hh.

References exists(), and idmap_m.

bool AMOS::Bank_t::existsIID ( ID_t  iid  )  const [inline]

Returns true if IID exists in the bank.

Definition at line 876 of file Bank_AMOS.hh.

References exists(), and idmap_m.

void AMOS::Bank_t::fetch ( const std::string &  eid,
IBankable_t &  obj 
) [inline]

Fetches a Bankable object from the bank by its EID.

Reimplemented in AMOS::BankStream_t.

Definition at line 909 of file Bank_AMOS.hh.

References fetchBID(), idmap_m, lookupBID(), and lookupIID().

void AMOS::Bank_t::fetch ( ID_t  iid,
IBankable_t &  obj 
) [inline]

Fetches a Bankable object from the bank by its IID.

Retrieves an object from the bank by its IID and stores it in a bankable object.

Parameters:
iid The IID of the object to fetch
obj A Bankable object to store the data
Precondition:
The bank is open for reading

The requested IID exists in the bank

obj is compatible with the current NCode bank type

Postcondition:
The desired object data will be loaded into obj
Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 898 of file Bank_AMOS.hh.

References fetchBID(), idmap_m, lookupBID(), and lookupEID().

Referenced by AMOS::BankStream_t::fetch().

void Bank_t::fetchBID ( ID_t  bid,
IBankable_t &  obj 
) [protected]

Fetch an object by BID.

Definition at line 500 of file Bank_AMOS.cc.

References AMOS_THROW_ARGUMENT, AMOS_THROW_IO, AMOS::B_READ, banktype_m, AMOS::Bank_t::BankPartition_t::fix, fix_size_m, AMOS::IBankable_t::flags_m, AMOS::IBankable_t::getNCode(), is_open_m, localizeBID(), mode_m, AMOS::readLE(), AMOS::IBankable_t::readRecord(), and AMOS::Bank_t::BankPartition_t::var.

Referenced by fetch().

void Bank_t::fetchBIDFix ( ID_t  iid,
IBankable_t &  obj 
) [protected]

Fetch the fixed record for an object by BID.

Definition at line 524 of file Bank_AMOS.cc.

References AMOS_THROW_ARGUMENT, AMOS_THROW_IO, AMOS::B_READ, banktype_m, AMOS::Bank_t::BankPartition_t::fix, fix_size_m, AMOS::IBankable_t::flags_m, AMOS::IBankable_t::getNCode(), is_open_m, localizeBID(), mode_m, AMOS::readLE(), and AMOS::IBankable_t::readRecordFix().

Referenced by fetchFix().

void AMOS::Bank_t::fetchFix ( const std::string &  eid,
IBankable_t &  obj 
) [inline]

Fetches the fixed length part of a Bankable object by its EID.

Reimplemented in AMOS::BankStream_t.

Definition at line 931 of file Bank_AMOS.hh.

References fetchBIDFix(), idmap_m, lookupBID(), and lookupIID().

void AMOS::Bank_t::fetchFix ( ID_t  iid,
IBankable_t &  obj 
) [inline]

Fetches the fixed length part of a Bankable object by its IID.

Reimplemented in AMOS::BankStream_t.

Definition at line 920 of file Bank_AMOS.hh.

References fetchBIDFix(), idmap_m, lookupBID(), and lookupEID().

Referenced by AMOS::BankStream_t::fetchFix().

const IDMap_t& AMOS::Bank_t::getIDMap (  )  const [inline]

Get the current (IID <-> EID) -> BID map for the bank.

Access to the bank's IDMap. One may iterate through the IDMap for a list of IDs in the bank. Users may not however directly modify the IDMap, because the bank will automatically update the IDMap based on the IDs of the objects appended to it. See IDMap_t API for more available methods.

Returns:
A const reference to the bank's IDMap

Definition at line 949 of file Bank_AMOS.hh.

References idmap_m.

Referenced by AMOS::BankStream_t::clean(), AMOS::BankStream_t::concat(), concat(), and AMOS::BankStream_t::open().

Size_t AMOS::Bank_t::getIDMapSize (  )  const [inline]

Get the number of objects in the Bank with and IID or EID.

Returns the total number of ID'ed objects stored in the bank. This number only includes object which have an associated IID or EID. Use getSize() to get the total number of objects in the bank.

Returns:
The number of ID'ed objects stored in the bank

Definition at line 964 of file Bank_AMOS.hh.

References getSize(), and idmap_m.

Size_t AMOS::Bank_t::getIndexSize (  )  const [inline]

Get the number of indices used by the bank.

Returns the number of BIDs (bank indices) being used by the bank. If we imagine the bank as an array, this would be the size of the array. In a perfect world, index size would equal the size of the bank (i.e. number of records in the bank), however when objects are deleted, instead of resizing the entire 'array' the index positions are ignored until a clean is issued. Thus, depending on the number of remove operations performed, the index size will be greater than or equal to the size of the bank. The bigger the gap, the more empty indices there will be and the more efficiency problems the bank will suffer. A clean operation will fix all this, and make index-size equal size.

Returns:
The number of indices being used

Definition at line 986 of file Bank_AMOS.hh.

References last_bid_m.

BankPartition_t* AMOS::Bank_t::getLastPartition (  )  [inline, protected]

Same as getPartition, but returns the current final partition.

Definition at line 452 of file Bank_AMOS.hh.

References getPartition(), and npartitions_m.

Referenced by appendBID(), concat(), and AMOS::BankStream_t::operator<<().

ID_t Bank_t::getMaxBID (  )  const

Searches through the bank for the largest BID.

Note:
O(n) time complexity
Returns:
The maximum BID in the bank

Definition at line 560 of file Bank_AMOS.cc.

References AMOS::IDMap_t::begin(), AMOS::IDMap_t::end(), idmap_m, and AMOS::NULL_ID.

ID_t Bank_t::getMaxIID (  )  const

Searches through the bank for the largest IID.

Note:
O(n) time complexity
Returns:
The maximum IID in the bank

Definition at line 548 of file Bank_AMOS.cc.

References AMOS::IDMap_t::begin(), AMOS::IDMap_t::end(), idmap_m, and AMOS::NULL_ID.

BankPartition_t* AMOS::Bank_t::getPartition ( ID_t  id  )  [inline, protected]

Returns the requested BankPartition, opening it if necessary.

Returns the requested (pre-existing) partition, opening it if necessary.

Parameters:
id The ID of the requested partition
Precondition:
There are adequate permissions in the bank directory

The requested partition is within range (not checked)

Postcondition:
The requested partition is open and on the queue

A previously opened partition may have been closed to make room

Exceptions:
IOException_t 
ArgumentException_t 
Returns:
The requested, opened partition

Definition at line 436 of file Bank_AMOS.hh.

References openPartition(), and partitions_m.

Referenced by concat(), getLastPartition(), and localizeBID().

Size_t AMOS::Bank_t::getSize (  )  const [inline]

Get the size of the bank, i.e. the number of stored records.

This method returns the total number of objects stored in the bank, including anonymous objects which have no associated IID or EID.

Returns:
The size of the bank, or 0 if the bank is empty

Definition at line 1021 of file Bank_AMOS.hh.

References nbids_m.

Referenced by empty(), and getIDMapSize().

signed char AMOS::Bank_t::getStatus (  )  const [inline]

Get the bank status.

Currently the status is set by the user and not written to the bank. So don't expect any interesting return values from this method. Status is set to zero on object initialization and then left untouched.

Returns:
Return the status of the bank

Definition at line 1036 of file Bank_AMOS.hh.

References status_m.

NCode_t AMOS::Bank_t::getType (  )  const [inline]

Get the unique bank type identifier.

Is equivalent to the NCode of the objects stored in the bank.

Returns:
The unique bank type identifier

Definition at line 1049 of file Bank_AMOS.hh.

References banktype_m.

void Bank_t::init (  )  [protected]

Initializes bank variables.

Reimplemented in AMOS::BankStream_t.

Definition at line 601 of file Bank_AMOS.cc.

References banktype_m, AMOS::IDMap_t::clear(), fix_size_m, idmap_m, is_open_m, last_bid_m, max_bid_m, nbids_m, npartitions_m, AMOS::NULL_ID, opened_m, partition_size_m, partitions_m, AMOS::IDMap_t::setType(), store_dir_m, and store_pfx_m.

Referenced by Bank_t(), close(), and destroy().

bool AMOS::Bank_t::isOpen (  )  const [inline]

Check the bank's open status.

Returns:
true if open, false if closed

Definition at line 1060 of file Bank_AMOS.hh.

References is_open_m.

Referenced by clean().

BankPartition_t* AMOS::Bank_t::localizeBID ( ID_t &  bid  )  [inline, protected]

Gets the partition and local identifier.

Parameters:
bid Lookup the location of this BID (1 based index))
Precondition:
bid is within range (not checked)
Postcondition:
bid will be adjusted to reference the returned partition
Returns:
The opened bank partition

Definition at line 472 of file Bank_AMOS.hh.

References getPartition(), and partition_size_m.

Referenced by fetchBID(), fetchBIDFix(), AMOS::BankStream_t::ignore(), AMOS::BankStream_t::operator>>(), removeBID(), and replaceBID().

void Bank_t::lockIFO (  )  [protected]

Obtains a file lock on the info store of the current bank.

Obtains a file lock on the info store of the current bank. Will throw an exception if the lock failed either because the info store does not exist or it took too long to obtain the lock. Has no effect if BankMode is set to B_SPY.

Postcondition:
The info store is locked
Exceptions:
IOException_t 
Returns:
void

Definition at line 620 of file Bank_AMOS.cc.

References AMOS_THROW_IO, AMOS::B_SPY, IFO_STORE_SUFFIX, LCK_STORE_SUFFIX, LOCK_TIME, mode_m, and store_pfx_m.

Referenced by syncIFO().

ID_t AMOS::Bank_t::lookupBID ( const std::string &  eid  )  const

Converts an EID to a BID, throw exception on failure.

Exceptions:
ArgumentException_t 
Returns:
The BID of the specified EID

ID_t Bank_t::lookupBID ( ID_t  iid  )  const

Converts an IID to a BID, throw exception on failure.

Exceptions:
ArgumentException_t 
Returns:
The BID of the specified IID

Definition at line 587 of file Bank_AMOS.cc.

References AMOS_THROW_ARGUMENT, idmap_m, last_bid_m, AMOS::IDMap_t::lookupBID(), and AMOS::NULL_ID.

Referenced by fetch(), fetchFix(), AMOS::BankStream_t::remove(), remove(), AMOS::BankStream_t::replace(), and replace().

const std::string& AMOS::Bank_t::lookupEID ( ID_t  iid  )  const [inline]

Converts an IID to an EID.

Returns:
Returns the EID of the IID or empty string on failure

Definition at line 1089 of file Bank_AMOS.hh.

References idmap_m.

Referenced by fetch(), and fetchFix().

ID_t AMOS::Bank_t::lookupIID ( const std::string &  eid  )  const [inline]

Converts an EID to an IID.

Returns:
Returns the IID of the EID or NULL_ID on failure

Definition at line 1100 of file Bank_AMOS.hh.

References idmap_m.

Referenced by fetch(), and fetchFix().

void AMOS::Bank_t::open ( const std::string &  dir,
BankMode_t  mode = B_READ|B_WRITE 
)

Opens a bank on disk.

Opens a bank on disk, allowing modification/access operations like append and fetch to be performed. An open bank will first be closed before the new one is opened. Check for the existence of a bank (with the exists method) before opening to avoid an exception. If the B_SPY mode is activated, only read access to the banks is required, otherwise both read and write access is required.

Parameters:
dir The resident directory of the bank
mode The mode of the bank (B_READ | B_WRITE | B_SPY)
Precondition:
At least one of the modes is specified

The specified directory contains a bank of this type

sufficient read/write/exe permissions for dir and bank files

Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Referenced by AMOS::BankStream_t::open().

Bank_t::BankPartition_t * Bank_t::openPartition ( ID_t  id  )  [protected]

Do not use this function, use getPartition instead.

Definition at line 676 of file Bank_AMOS.cc.

References AMOS_THROW_IO, AMOS::Bank_t::BankPartition_t::fix, AMOS::Bank_t::BankPartition_t::fix_name, max_partitions_m, mode_m, opened_m, partitions_m, AMOS::Bank_t::BankPartition_t::var, and AMOS::Bank_t::BankPartition_t::var_name.

Referenced by getPartition().

Bank_t& AMOS::Bank_t::operator= ( const Bank_t &  source  )  [protected]

Assignment (copy) operator.

Private as to disallow copying

void AMOS::Bank_t::remove ( const std::string &  eid  )  [inline]

Removes an object from the bank by its EID.

Reimplemented in AMOS::BankStream_t.

Definition at line 1155 of file Bank_AMOS.hh.

References idmap_m, lookupBID(), remove(), and removeBID().

void AMOS::Bank_t::remove ( ID_t  iid  )  [inline]

Removes an object from the bank by its IID.

Removes the object from the bank. Removed objects will no longer be accessible and the EID/IID pair will be removed from the IDMap. However, removing an object does not free the disk space the object was occupying so an eventual clean operation may be necessary to reduce the physical size of the bank.

Parameters:
iid The IID of the object to remove
Precondition:
The bank is open for read/writing

The IID exists in the bank

Postcondition:
The object is conceptually removed from the bank
Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 1145 of file Bank_AMOS.hh.

References idmap_m, lookupBID(), and removeBID().

Referenced by remove().

void Bank_t::removeBID ( ID_t  bid  )  [protected]

Remove an object by BID.

Definition at line 717 of file Bank_AMOS.cc.

References AMOS_THROW_IO, AMOS::B_READ, AMOS::Bank_t::BankPartition_t::fix, fix_size_m, is_open_m, AMOS::BankFlags_t::is_removed, localizeBID(), mode_m, nbids_m, AMOS::readLE(), AMOS::Bank_t::BankPartition_t::var, and AMOS::writeLE().

Referenced by AMOS::BankStream_t::remove(), remove(), and AMOS::BankStream_t::removeByBID().

void AMOS::Bank_t::replace ( const std::string &  eid,
IBankable_t &  obj 
)

Replaces an object in the bank by its EID.

Reimplemented in AMOS::BankStream_t.

void Bank_t::replace ( ID_t  iid,
IBankable_t &  obj 
)

Replaces an object in the bank by its IID.

Replaces an object in the bank with a given IID. EID/IID pair of the replaced object will be removed from the IDMap and the new pair will be added. Replaced object will not be fetchable or restoreable, it will be overwritten by the new object. The removed flag of the object will be cleared but the modified flag will be set. The disk space of the replaced object is not freed, therefore an eventual clean operation may be necessary to reduce the physical size of the bank.

Parameters:
iid The IID of the object to replace
obj The Bankable object to replace old object
Precondition:
The bank is open for read/writing

The specified IID exists in the bank

obj is compatible with the current NCode bank type

The new object EID does not exist in the bank

Postcondition:
obj will replace the object with the given IID

obj's is_modified flag is set

Exceptions:
IOException_t 
ArgumentException_t 
Returns:
void

Reimplemented in AMOS::BankStream_t.

Definition at line 741 of file Bank_AMOS.cc.

References AMOS::IBankable_t::eid_m, idmap_m, AMOS::IBankable_t::iid_m, AMOS::IDMap_t::insert(), lookupBID(), AMOS::IDMap_t::lookupEID(), AMOS::IDMap_t::remove(), and replaceBID().

void Bank_t::replaceBID ( ID_t  bid,
IBankable_t &  obj 
) [protected]

Replace an object by BID.

Definition at line 793 of file Bank_AMOS.cc.

References AMOS_THROW_ARGUMENT, AMOS_THROW_IO, AMOS::B_READ, banktype_m, AMOS::Bank_t::BankPartition_t::fix, fix_size_m, AMOS::IBankable_t::flags_m, AMOS::IBankable_t::getNCode(), AMOS::BankFlags_t::is_modified, is_open_m, AMOS::BankFlags_t::is_removed, localizeBID(), mode_m, AMOS::Bank_t::BankPartition_t::var, AMOS::writeLE(), and AMOS::IBankable_t::writeRecord().

Referenced by AMOS::BankStream_t::replace(), replace(), and AMOS::BankStream_t::replaceByBID().

void AMOS::Bank_t::setMode ( BankMode_t  mode  )  [inline, protected]

validates and sets the BankMode

Parameters:
mode BankMode_t to check and set
Precondition:
mode is a valid BankMode_t
Exceptions:
ArgumentException_t 
Returns:
void

Definition at line 521 of file Bank_AMOS.hh.

References AMOS_THROW_ARGUMENT, AMOS::B_READ, AMOS::B_SPY, AMOS::B_WRITE, and mode_m.

void AMOS::Bank_t::setStatus ( signed char  status  )  [inline]

Set the bank status.

Currently the status is set by the user and not written to the bank. Status is set to zero on object initialization and then left untouched.

Returns:
void

Definition at line 1202 of file Bank_AMOS.hh.

References status_m.

void Bank_t::syncIFO ( IFOMode_t  mode  )  [protected]

Syncs the IFO store with in-memory data.

Locks the IFO store if needed, then syncs the IFO store with in-memory data. Either as an open, update or close depending on the IFOMode. Use I_OPEN if opening the IFO store for the first time, I_CREATE if creating a new IFO store or I_CLOSE if ready to close the IFO store. Only I_OPEN will have an effect if bank is in B_SPY mode. Will throw an exception if any of the bank locks are violated.

Precondition:
The bank is open
Parameters:
mode I_OPEN, I_CREATE or I_CLOSE (cannot OR these together)
Exceptions:
IOException_t 
Returns:
void

Definition at line 823 of file Bank_AMOS.cc.

References addPartition(), AMOS_THROW_IO, AMOS::B_READ, AMOS::B_SPY, BANK_VERSION, banktype_m, AMOS::Decode(), fix_size_m, I_CLOSE, I_OPEN, IFO_STORE_SUFFIX, last_bid_m, lockIFO(), mode_m, nbids_m, npartitions_m, AMOS::NULL_CHAR, partition_size_m, partitions_m, READ_LOCK_CHAR, store_pfx_m, unlockIFO(), and WRITE_LOCK_CHAR.

Referenced by close().

void AMOS::Bank_t::touchFile ( const std::string &  path,
int  mode,
bool  create 
) [protected]

Opens or creates a file, throwing exception on failure.

Referenced by addPartition().

void Bank_t::unlockIFO (  )  [protected]

Releases the file lock on the info store of the current bank.

Releases the file lock on the info store of the current bank. Will throw an exception if the unlock failed either because the lock did not exist or could not be released. Has no effect if BankMode is set to B_SPY.

Precondition:
The bank is open

The info store is currently locked

Exceptions:
IOException_t 
Returns:
void

Definition at line 1048 of file Bank_AMOS.cc.

References AMOS_THROW_IO, LCK_STORE_SUFFIX, mode_m, and store_pfx_m.

Referenced by syncIFO().

Member Data Documentation

const string Bank_t::BANK_VERSION = "2.8" [static]

current bank version

Definition at line 619 of file Bank_AMOS.hh.

Referenced by AMOS::PrintBankVersion(), and syncIFO().

NCode_t AMOS::Bank_t::banktype_m [protected]

the type of objects stored in this bank

Definition at line 591 of file Bank_AMOS.hh.

Referenced by appendBID(), clean(), clear(), concat(), fetchBID(), fetchBIDFix(), getType(), init(), AMOS::BankStream_t::operator<<(), AMOS::BankStream_t::operator>>(), replaceBID(), and syncIFO().

Size_t AMOS::Bank_t::buffer_size_m [protected]

size of the I/O buffer

Definition at line 593 of file Bank_AMOS.hh.

Referenced by addPartition(), Bank_t(), and AMOS::BankStream_t::BankStream_t().

const Size_t Bank_t::DEFAULT_BUFFER_SIZE = 1024 [static, protected]

IO buffer size.

Reimplemented in AMOS::BankStream_t.

Definition at line 338 of file Bank_AMOS.hh.

Referenced by Bank_t().

const Size_t Bank_t::DEFAULT_PARTITION_SIZE = 1000000 [static, protected]

records per partition

Definition at line 339 of file Bank_AMOS.hh.

Referenced by addPartition().

Size_t AMOS::Bank_t::fix_size_m [protected]

size of entire fixed length record

Definition at line 602 of file Bank_AMOS.hh.

Referenced by appendBID(), clean(), concat(), fetchBID(), fetchBIDFix(), AMOS::BankStream_t::ignore(), init(), AMOS::BankStream_t::operator<<(), AMOS::BankStream_t::operator>>(), removeBID(), replaceBID(), and syncIFO().

const string Bank_t::FIX_STORE_SUFFIX = ".fix" [static]

the fixed length stores

Definition at line 625 of file Bank_AMOS.hh.

Referenced by addPartition().

IDMap_t AMOS::Bank_t::idmap_m [protected]

the IDMap IID <-> EID to BID

Definition at line 612 of file Bank_AMOS.hh.

Referenced by append(), clean(), clear(), close(), concat(), existsEID(), existsIID(), fetch(), fetchFix(), getIDMap(), getIDMapSize(), getMaxBID(), getMaxIID(), init(), lookupBID(), lookupEID(), lookupIID(), AMOS::BankStream_t::operator<<(), AMOS::BankStream_t::remove(), remove(), AMOS::BankStream_t::removeByBID(), AMOS::BankStream_t::replace(), replace(), and AMOS::BankStream_t::replaceByBID().

const string Bank_t::IFO_STORE_SUFFIX = ".ifo" [static]

the informational store

Definition at line 621 of file Bank_AMOS.hh.

Referenced by AMOS::BankExists(), destroy(), lockIFO(), and syncIFO().

bool AMOS::Bank_t::is_open_m [protected]

open status of the bank

Definition at line 596 of file Bank_AMOS.hh.

Referenced by appendBID(), clean(), clear(), close(), concat(), destroy(), AMOS::BankStream_t::eof(), fetchBID(), fetchBIDFix(), AMOS::BankStream_t::ignore(), init(), isOpen(), AMOS::BankStream_t::operator<<(), AMOS::BankStream_t::operator>>(), removeBID(), replaceBID(), AMOS::BankStream_t::seekg(), ~Bank_t(), and AMOS::BankStream_t::~BankStream_t().

ID_t AMOS::Bank_t::last_bid_m [protected]

the last bank bid (1 based)

Definition at line 605 of file Bank_AMOS.hh.

Referenced by append(), appendBID(), AMOS::BankStream_t::clean(), clean(), clear(), AMOS::BankStream_t::concat(), concat(), getIndexSize(), init(), AMOS::BankStream_t::inrange(), lookupBID(), AMOS::BankStream_t::open(), AMOS::BankStream_t::operator<<(), AMOS::BankStream_t::removeByBID(), AMOS::BankStream_t::replaceByBID(), AMOS::BankStream_t::seekg(), syncIFO(), and AMOS::BankStream_t::tellp().

const string Bank_t::LCK_STORE_SUFFIX = ".lck" [static]

the ifo store file lock

Definition at line 623 of file Bank_AMOS.hh.

Referenced by destroy(), lockIFO(), and unlockIFO().

const string Bank_t::MAP_STORE_SUFFIX = ".map" [static]

the ID map store

Definition at line 622 of file Bank_AMOS.hh.

Referenced by close(), and destroy().

ID_t AMOS::Bank_t::max_bid_m [protected]

maximum bid given the current partitioning

Definition at line 606 of file Bank_AMOS.hh.

Referenced by addPartition(), appendBID(), clear(), concat(), init(), and AMOS::BankStream_t::operator<<().

const Size_t Bank_t::MAX_OPEN_PARTITIONS = 20 [static, protected]

open partitions

Reimplemented in AMOS::BankStream_t.

Definition at line 340 of file Bank_AMOS.hh.

Referenced by Bank_t().

Size_t AMOS::Bank_t::max_partitions_m [protected]

maximum number of open partitions

Definition at line 594 of file Bank_AMOS.hh.

Referenced by Bank_t(), AMOS::BankStream_t::BankStream_t(), and openPartition().

BankMode_t AMOS::Bank_t::mode_m [protected]

mode of the bank, B_READ|B_WRITE|B_SPY

Definition at line 598 of file Bank_AMOS.hh.

Referenced by appendBID(), clean(), clear(), close(), concat(), destroy(), fetchBID(), fetchBIDFix(), AMOS::BankStream_t::ignore(), lockIFO(), openPartition(), AMOS::BankStream_t::operator<<(), AMOS::BankStream_t::operator>>(), removeBID(), replaceBID(), AMOS::BankStream_t::seekg(), setMode(), syncIFO(), and unlockIFO().

ID_t AMOS::Bank_t::nbids_m [protected]

number of non-deleted bids

Definition at line 607 of file Bank_AMOS.hh.

Referenced by appendBID(), clean(), clear(), concat(), getSize(), init(), AMOS::BankStream_t::operator<<(), removeBID(), and syncIFO().

Size_t AMOS::Bank_t::npartitions_m [protected]

number of partitions

Definition at line 608 of file Bank_AMOS.hh.

Referenced by addPartition(), clean(), clear(), close(), concat(), getLastPartition(), init(), and syncIFO().

std::deque<BankPartition_t *> AMOS::Bank_t::opened_m [protected]

opened partitions

Definition at line 609 of file Bank_AMOS.hh.

Referenced by clear(), init(), and openPartition().

Size_t AMOS::Bank_t::partition_size_m [protected]

records per disk store partition

Definition at line 603 of file Bank_AMOS.hh.

Referenced by addPartition(), clean(), clear(), init(), localizeBID(), and syncIFO().

std::vector<BankPartition_t *> AMOS::Bank_t::partitions_m [protected]

all partitions

Definition at line 610 of file Bank_AMOS.hh.

Referenced by addPartition(), clean(), clear(), close(), getPartition(), init(), openPartition(), and syncIFO().

const char Bank_t::READ_LOCK_CHAR = 'r' [static]

read lock char

Definition at line 631 of file Bank_AMOS.hh.

Referenced by syncIFO().

signed char AMOS::Bank_t::status_m [protected]

bank status

Definition at line 597 of file Bank_AMOS.hh.

Referenced by Bank_t(), getStatus(), and setStatus().

std::string AMOS::Bank_t::store_dir_m [protected]

the disk store directory

Definition at line 600 of file Bank_AMOS.hh.

Referenced by destroy(), and init().

std::string AMOS::Bank_t::store_pfx_m [protected]

the disk store prefix (including dir)

Definition at line 601 of file Bank_AMOS.hh.

Referenced by addPartition(), clean(), close(), destroy(), init(), lockIFO(), syncIFO(), and unlockIFO().

const string Bank_t::TMP_STORE_SUFFIX = ".tmp" [static]

the temporary store

Definition at line 628 of file Bank_AMOS.hh.

Referenced by clean().

const string Bank_t::VAR_STORE_SUFFIX = ".var" [static]

the variable length stores

Definition at line 626 of file Bank_AMOS.hh.

Referenced by addPartition().

const char Bank_t::WRITE_LOCK_CHAR = 'w' [static]

write lock char

Definition at line 630 of file Bank_AMOS.hh.

Referenced by syncIFO().

The documentation for this class was generated from the following files:
Generated on Mon Feb 22 17:36:34 2010 for libAMOS by  doxygen 1.4.7