public class BookKeeper extends Object implements AutoCloseable
We assume there is one single writer to a ledger at any time.
There are four possible operations: start a new ledger, write to a ledger, read from a ledger and delete a ledger.
The exceptions resulting from synchronous calls and error code resulting from
asynchronous calls can be found in the class BKException
.
Modifier and Type | Class and Description |
---|---|
static class |
BookKeeper.Builder
BookKeeper Client Builder to build client instances.
|
static class |
BookKeeper.DigestType
There are 2 digest types that can be used for verification.
|
(package private) static class |
BookKeeper.SyncOpenCallback |
Modifier and Type | Field and Description |
---|---|
(package private) org.apache.bookkeeper.proto.BookieClient |
bookieClient |
(package private) BookieInfoReader |
bookieInfoReader |
(package private) ScheduledExecutorService |
bookieInfoScheduler |
(package private) BookieWatcher |
bookieWatcher |
(package private) boolean |
closed |
(package private) ReentrantReadWriteLock |
closeLock |
(package private) ClientConfiguration |
conf |
(package private) boolean |
delayEnsembleChange |
(package private) Feature |
disableEnsembleChangeFeature |
(package private) io.netty.channel.EventLoopGroup |
eventLoopGroup |
(package private) int |
explicitLacInterval |
(package private) FeatureProvider |
featureProvider |
(package private) org.apache.bookkeeper.meta.LedgerIdGenerator |
ledgerIdGenerator |
(package private) org.apache.bookkeeper.meta.LedgerManager |
ledgerManager |
(package private) org.apache.bookkeeper.meta.LedgerManagerFactory |
ledgerManagerFactory |
(package private) static org.slf4j.Logger |
LOG |
(package private) org.apache.bookkeeper.util.OrderedSafeExecutor |
mainWorkerPool |
(package private) boolean |
ownEventLoopGroup |
(package private) boolean |
ownTimer |
(package private) boolean |
ownZKHandle |
(package private) EnsemblePlacementPolicy |
placementPolicy |
(package private) com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> |
readLACSpeculativeRequestPolicy |
(package private) com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> |
readSpeculativeRequestPolicy |
(package private) boolean |
reorderReadSequence |
(package private) io.netty.util.HashedWheelTimer |
requestTimer |
(package private) ScheduledExecutorService |
scheduler |
(package private) org.apache.zookeeper.ZooKeeper |
zk |
Constructor and Description |
---|
BookKeeper(ClientConfiguration conf)
Create a bookkeeper client using a configuration object.
|
BookKeeper(ClientConfiguration conf,
org.apache.zookeeper.ZooKeeper zk)
Create a bookkeeper client but use the passed in zookeeper client instead
of instantiating one.
|
BookKeeper(ClientConfiguration conf,
org.apache.zookeeper.ZooKeeper zk,
io.netty.channel.EventLoopGroup eventLoopGroup)
Create a bookkeeper client but use the passed in zookeeper client and
client event loop group instead of instantiating those.
|
BookKeeper(String servers)
Create a bookkeeper client.
|
Modifier and Type | Method and Description |
---|---|
void |
asyncCreateLedger(int ensSize,
int writeQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
AsyncCallback.CreateCallback cb,
Object ctx)
Creates a new ledger asynchronously.
|
void |
asyncCreateLedger(int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
AsyncCallback.CreateCallback cb,
Object ctx,
Map<String,byte[]> customMetadata)
Creates a new ledger asynchronously.
|
void |
asyncCreateLedgerAdv(int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
AsyncCallback.CreateCallback cb,
Object ctx,
Map<String,byte[]> customMetadata)
Creates a new ledger asynchronously and returns
LedgerHandleAdv
which can accept entryId. |
void |
asyncCreateLedgerAdv(long ledgerId,
int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
AsyncCallback.CreateCallback cb,
Object ctx,
Map<String,byte[]> customMetadata)
Asynchronously creates a new ledger using the interface which accepts a ledgerId as input.
|
void |
asyncDeleteLedger(long lId,
AsyncCallback.DeleteCallback cb,
Object ctx)
Deletes a ledger asynchronously.
|
void |
asyncIsClosed(long lId,
AsyncCallback.IsClosedCallback cb,
Object ctx)
Check asynchronously whether the ledger with identifier lId
has been closed.
|
void |
asyncOpenLedger(long lId,
BookKeeper.DigestType digestType,
byte[] passwd,
AsyncCallback.OpenCallback cb,
Object ctx)
Open existing ledger asynchronously for reading.
|
void |
asyncOpenLedgerNoRecovery(long lId,
BookKeeper.DigestType digestType,
byte[] passwd,
AsyncCallback.OpenCallback cb,
Object ctx)
Open existing ledger asynchronously for reading, but it does not try to
recover the ledger if it is not yet closed.
|
(package private) void |
checkForFaultyBookies() |
void |
close()
Shuts down client.
|
LedgerHandle |
createLedger(BookKeeper.DigestType digestType,
byte[] passwd)
Creates a new ledger.
|
LedgerHandle |
createLedger(int ensSize,
int qSize,
BookKeeper.DigestType digestType,
byte[] passwd)
Synchronous call to create ledger.
|
LedgerHandle |
createLedger(int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd)
Synchronous call to create ledger.
|
LedgerHandle |
createLedger(int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
Map<String,byte[]> customMetadata)
Synchronous call to create ledger.
|
LedgerHandle |
createLedgerAdv(int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd)
Synchronous call to create ledger.
|
LedgerHandle |
createLedgerAdv(int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
Map<String,byte[]> customMetadata)
Synchronous call to create ledger.
|
LedgerHandle |
createLedgerAdv(long ledgerId,
int ensSize,
int writeQuorumSize,
int ackQuorumSize,
BookKeeper.DigestType digestType,
byte[] passwd,
Map<String,byte[]> customMetadata)
Synchronously creates a new ledger using the interface which accepts a ledgerId as input.
|
void |
deleteLedger(long lId)
Synchronous call to delete a ledger.
|
static BookKeeper.Builder |
forConfig(ClientConfiguration conf) |
(package private) OpStatsLogger |
getAddOpLogger() |
(package private) org.apache.bookkeeper.proto.BookieClient |
getBookieClient()
Get the BookieClient, currently used for doing bookie recovery.
|
Map<org.apache.bookkeeper.net.BookieSocketAddress,BookieInfoReader.BookieInfo> |
getBookieInfo()
Retrieves BookieInfo from all the bookies in the cluster.
|
protected ClientConfiguration |
getConf() |
(package private) OpStatsLogger |
getCreateOpLogger() |
(package private) static io.netty.channel.EventLoopGroup |
getDefaultEventLoopGroup() |
(package private) OpStatsLogger |
getDeleteOpLogger() |
int |
getExplicitLacInterval() |
(package private) org.apache.bookkeeper.meta.LedgerIdGenerator |
getLedgerIdGenerator() |
(package private) org.apache.bookkeeper.meta.LedgerManager |
getLedgerManager() |
(package private) OpStatsLogger |
getOpenOpLogger() |
(package private) OpStatsLogger |
getReadLacAndEntryOpLogger() |
(package private) OpStatsLogger |
getReadLacAndEntryRespLogger() |
(package private) OpStatsLogger |
getReadLacOpLogger() |
com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> |
getReadLACSpeculativeRequestPolicy() |
(package private) OpStatsLogger |
getReadOpLogger() |
com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> |
getReadSpeculativeRequestPolicy() |
(package private) OpStatsLogger |
getRecoverAddCountLogger() |
(package private) OpStatsLogger |
getRecoverOpLogger() |
(package private) OpStatsLogger |
getRecoverReadCountLogger() |
(package private) int |
getReturnRc(int rc) |
(package private) StatsLogger |
getStatsLogger() |
(package private) org.apache.bookkeeper.meta.LedgerManager |
getUnderlyingLedgerManager() |
(package private) OpStatsLogger |
getWriteLacOpLogger() |
(package private) org.apache.zookeeper.ZooKeeper |
getZkHandle() |
boolean |
isClosed(long lId)
Check whether the ledger with identifier lId
has been closed.
|
LedgerHandle |
openLedger(long lId,
BookKeeper.DigestType digestType,
byte[] passwd)
Synchronous open ledger call
|
LedgerHandle |
openLedgerNoRecovery(long lId,
BookKeeper.DigestType digestType,
byte[] passwd)
Synchronous, unsafe open ledger call
|
(package private) void |
scheduleBookieHealthCheckIfEnabled() |
static final org.slf4j.Logger LOG
final org.apache.zookeeper.ZooKeeper zk
final io.netty.channel.EventLoopGroup eventLoopGroup
boolean ownEventLoopGroup
boolean ownZKHandle
final org.apache.bookkeeper.proto.BookieClient bookieClient
final BookieWatcher bookieWatcher
final org.apache.bookkeeper.util.OrderedSafeExecutor mainWorkerPool
final ScheduledExecutorService scheduler
final io.netty.util.HashedWheelTimer requestTimer
final boolean ownTimer
final FeatureProvider featureProvider
final ScheduledExecutorService bookieInfoScheduler
final Feature disableEnsembleChangeFeature
final org.apache.bookkeeper.meta.LedgerManagerFactory ledgerManagerFactory
final org.apache.bookkeeper.meta.LedgerManager ledgerManager
final org.apache.bookkeeper.meta.LedgerIdGenerator ledgerIdGenerator
final EnsemblePlacementPolicy placementPolicy
BookieInfoReader bookieInfoReader
final ClientConfiguration conf
final int explicitLacInterval
final boolean delayEnsembleChange
final boolean reorderReadSequence
final com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> readSpeculativeRequestPolicy
final com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> readLACSpeculativeRequestPolicy
boolean closed
final ReentrantReadWriteLock closeLock
public BookKeeper(String servers) throws IOException, InterruptedException, org.apache.zookeeper.KeeperException
servers
- A list of one of more servers on which zookeeper is running. The
client assumes that the running bookies have been registered with
zookeeper under the path
BookieWatcher.bookieRegistrationPath
IOException
InterruptedException
org.apache.zookeeper.KeeperException
public BookKeeper(ClientConfiguration conf) throws IOException, InterruptedException, org.apache.zookeeper.KeeperException
conf
- Client Configuration objectIOException
InterruptedException
org.apache.zookeeper.KeeperException
public BookKeeper(ClientConfiguration conf, org.apache.zookeeper.ZooKeeper zk) throws IOException, InterruptedException, org.apache.zookeeper.KeeperException
conf
- Client Configuration object
ClientConfiguration
zk
- Zookeeper client instance connected to the zookeeper with which
the bookies have registeredIOException
InterruptedException
org.apache.zookeeper.KeeperException
public BookKeeper(ClientConfiguration conf, org.apache.zookeeper.ZooKeeper zk, io.netty.channel.EventLoopGroup eventLoopGroup) throws IOException, InterruptedException, org.apache.zookeeper.KeeperException
conf
- Client Configuration Object
ClientConfiguration
zk
- Zookeeper client instance connected to the zookeeper with which
the bookies have registered. The ZooKeeper client must be connected
before it is passed to BookKeeper. Otherwise a KeeperException is thrown.eventLoopGroup
- An event loop group that will be used to create connections to the bookiesIOException
InterruptedException
org.apache.zookeeper.KeeperException
- if the passed zk handle is not connectedpublic static BookKeeper.Builder forConfig(ClientConfiguration conf)
public int getExplicitLacInterval()
int getReturnRc(int rc)
void scheduleBookieHealthCheckIfEnabled()
void checkForFaultyBookies()
org.apache.bookkeeper.meta.LedgerManager getLedgerManager()
org.apache.bookkeeper.meta.LedgerManager getUnderlyingLedgerManager()
org.apache.bookkeeper.meta.LedgerIdGenerator getLedgerIdGenerator()
org.apache.zookeeper.ZooKeeper getZkHandle()
protected ClientConfiguration getConf()
StatsLogger getStatsLogger()
public com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> getReadSpeculativeRequestPolicy()
public com.google.common.base.Optional<SpeculativeRequestExecutionPolicy> getReadLACSpeculativeRequestPolicy()
org.apache.bookkeeper.proto.BookieClient getBookieClient()
public Map<org.apache.bookkeeper.net.BookieSocketAddress,BookieInfoReader.BookieInfo> getBookieInfo() throws BKException, InterruptedException
BKException
InterruptedException
public void asyncCreateLedger(int ensSize, int writeQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, AsyncCallback.CreateCallback cb, Object ctx)
ensSize
- number of bookies over which to stripe entrieswriteQuorumSize
- number of bookies each entry will be written to. each of these bookies
must acknowledge the entry before the call is completed.digestType
- digest type, either MAC or CRC32passwd
- passwordcb
- createCallback implementationctx
- optional control objectpublic void asyncCreateLedger(int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, AsyncCallback.CreateCallback cb, Object ctx, Map<String,byte[]> customMetadata)
ClientConfiguration.getReadTimeout()
. Once the bookie failure is detected,
that bookie will be removed from the ensemble.
The other parameters match those of asyncCreateLedger(int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- number of bookies over which to stripe entrieswriteQuorumSize
- number of bookies each entry will be written toackQuorumSize
- number of bookies which must acknowledge an entry before the call is completeddigestType
- digest type, either MAC or CRC32passwd
- passwordcb
- createCallback implementationctx
- optional control objectcustomMetadata
- optional customMetadata that holds user specified metadatapublic LedgerHandle createLedger(BookKeeper.DigestType digestType, byte[] passwd) throws BKException, InterruptedException
digestType
- digest type, either MAC or CRC32passwd
- passwordInterruptedException
BKException
public LedgerHandle createLedger(int ensSize, int qSize, BookKeeper.DigestType digestType, byte[] passwd) throws InterruptedException, BKException
asyncCreateLedger(int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- qSize
- digestType
- passwd
- InterruptedException
BKException
public LedgerHandle createLedger(int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd) throws InterruptedException, BKException
asyncCreateLedger(int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- writeQuorumSize
- ackQuorumSize
- digestType
- passwd
- InterruptedException
BKException
public LedgerHandle createLedger(int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, Map<String,byte[]> customMetadata) throws InterruptedException, BKException
#asyncCreateLedger(int, int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- writeQuorumSize
- ackQuorumSize
- digestType
- passwd
- customMetadata
- InterruptedException
BKException
public LedgerHandle createLedgerAdv(int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd) throws InterruptedException, BKException
LedgerHandleAdv
which can accept entryId.
Parameters must match those of
#asyncCreateLedgerAdv(int, int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- writeQuorumSize
- ackQuorumSize
- digestType
- passwd
- customMetadata
- InterruptedException
BKException
public LedgerHandle createLedgerAdv(int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, Map<String,byte[]> customMetadata) throws InterruptedException, BKException
LedgerHandleAdv
which can accept entryId.
Parameters must match those of
#asyncCreateLedgerAdv(int, int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- writeQuorumSize
- ackQuorumSize
- digestType
- passwd
- customMetadata
- InterruptedException
BKException
public void asyncCreateLedgerAdv(int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, AsyncCallback.CreateCallback cb, Object ctx, Map<String,byte[]> customMetadata)
LedgerHandleAdv
which can accept entryId. Ledgers created with this call have ability to accept
a separate write quorum and ack quorum size. The write quorum must be larger than
the ack quorum.
Separating the write and the ack quorum allows the BookKeeper client to continue
writing when a bookie has failed but the failure has not yet been detected. Detecting
a bookie has failed can take a number of seconds, as configured by the read timeout
ClientConfiguration.getReadTimeout()
. Once the bookie failure is detected,
that bookie will be removed from the ensemble.
The other parameters match those of asyncCreateLedger(int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ensSize
- number of bookies over which to stripe entrieswriteQuorumSize
- number of bookies each entry will be written toackQuorumSize
- number of bookies which must acknowledge an entry before the call is completeddigestType
- digest type, either MAC or CRC32passwd
- passwordcb
- createCallback implementationctx
- optional control objectcustomMetadata
- optional customMetadata that holds user specified metadatapublic LedgerHandle createLedgerAdv(long ledgerId, int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, Map<String,byte[]> customMetadata) throws InterruptedException, BKException
LedgerHandleAdv
which can accept entryId.
Parameters must match those of
#asyncCreateLedgerAdvWithLedgerId(byte[], long, int, int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ledgerId
- ensSize
- writeQuorumSize
- ackQuorumSize
- digestType
- passwd
- customMetadata
- InterruptedException
BKException
public void asyncCreateLedgerAdv(long ledgerId, int ensSize, int writeQuorumSize, int ackQuorumSize, BookKeeper.DigestType digestType, byte[] passwd, AsyncCallback.CreateCallback cb, Object ctx, Map<String,byte[]> customMetadata)
LedgerHandleAdv
which can accept entryId.
Ledgers created with this call have ability to accept
a separate write quorum and ack quorum size. The write quorum must be larger than
the ack quorum.
Separating the write and the ack quorum allows the BookKeeper client to continue
writing when a bookie has failed but the failure has not yet been detected. Detecting
a bookie has failed can take a number of seconds, as configured by the read timeout
ClientConfiguration.getReadTimeout()
. Once the bookie failure is detected,
that bookie will be removed from the ensemble.
The other parameters match those of #asyncCreateLedger(long, int, int, DigestType, byte[],
AsyncCallback.CreateCallback, Object)
ledgerId
- ledger Id to use for the newly created ledgerensSize
- number of bookies over which to stripe entrieswriteQuorumSize
- number of bookies each entry will be written toackQuorumSize
- number of bookies which must acknowledge an entry before the call is completeddigestType
- digest type, either MAC or CRC32passwd
- passwordcb
- createCallback implementationctx
- optional control objectcustomMetadata
- optional customMetadata that holds user specified metadatapublic void asyncOpenLedger(long lId, BookKeeper.DigestType digestType, byte[] passwd, AsyncCallback.OpenCallback cb, Object ctx)
lId
- ledger identifierdigestType
- digest type, either MAC or CRC32passwd
- passwordctx
- optional control objectLedgerHandle.asyncClose(org.apache.bookkeeper.client.AsyncCallback.CloseCallback, java.lang.Object)
public void asyncOpenLedgerNoRecovery(long lId, BookKeeper.DigestType digestType, byte[] passwd, AsyncCallback.OpenCallback cb, Object ctx)
lId
- ledger identifierdigestType
- digest type, either MAC or CRC32passwd
- passwordctx
- optional control objectpublic LedgerHandle openLedger(long lId, BookKeeper.DigestType digestType, byte[] passwd) throws BKException, InterruptedException
lId
- ledger identifierdigestType
- digest type, either MAC or CRC32passwd
- passwordInterruptedException
BKException
asyncOpenLedger(long, org.apache.bookkeeper.client.BookKeeper.DigestType, byte[], org.apache.bookkeeper.client.AsyncCallback.OpenCallback, java.lang.Object)
public LedgerHandle openLedgerNoRecovery(long lId, BookKeeper.DigestType digestType, byte[] passwd) throws BKException, InterruptedException
lId
- ledger identifierdigestType
- digest type, either MAC or CRC32passwd
- passwordInterruptedException
BKException
asyncOpenLedgerNoRecovery(long, org.apache.bookkeeper.client.BookKeeper.DigestType, byte[], org.apache.bookkeeper.client.AsyncCallback.OpenCallback, java.lang.Object)
public void asyncDeleteLedger(long lId, AsyncCallback.DeleteCallback cb, Object ctx)
lId
- ledger Idcb
- deleteCallback implementationctx
- optional control objectpublic void deleteLedger(long lId) throws InterruptedException, BKException
asyncDeleteLedger(long, AsyncCallback.DeleteCallback, Object)
lId
- ledgerIdInterruptedException
BKException.BKNoSuchLedgerExistsException
- if the ledger doesn't existBKException
public void asyncIsClosed(long lId, AsyncCallback.IsClosedCallback cb, Object ctx)
lId
- ledger identifiercb
- callback methodpublic boolean isClosed(long lId) throws BKException, InterruptedException
lId
- BKException
InterruptedException
public void close() throws InterruptedException, BKException
close
in interface AutoCloseable
InterruptedException
BKException
OpStatsLogger getCreateOpLogger()
OpStatsLogger getOpenOpLogger()
OpStatsLogger getDeleteOpLogger()
OpStatsLogger getRecoverOpLogger()
OpStatsLogger getReadOpLogger()
OpStatsLogger getReadLacAndEntryOpLogger()
OpStatsLogger getReadLacAndEntryRespLogger()
OpStatsLogger getAddOpLogger()
OpStatsLogger getWriteLacOpLogger()
OpStatsLogger getReadLacOpLogger()
OpStatsLogger getRecoverAddCountLogger()
OpStatsLogger getRecoverReadCountLogger()
static io.netty.channel.EventLoopGroup getDefaultEventLoopGroup()
Copyright © 2011–2017 The Apache Software Foundation. All rights reserved.