Bitswap is a libp2p data exchange protocol for sending and receiving content addressed blocks of data.
Bitswap has two primary jobs:
Bitswap is a message-based protocol, as opposed to request-response. All messages
contain wantlists, and/or blocks. Upon receiving a wantlist, a Bitswap server SHOULD
eventually process and respond to the requester with either information about the
block or the block itself. Upon receiving blocks, the client SHOULD send a Cancel
notification to peers that have asked for the data, signifying that the client no
longer wants the block.
Bitswap aims to be a simple protocol, so that implementations can balance aspects such as throughput, latency, fairness, memory usage, etc. for their specific requirements.
There are multiple Bitswap versions and more may evolve over time. We give brief overviews as to the changes behind each protocol version:
/ipfs/bitswap/1.0.0
- Initial version/ipfs/bitswap/1.1.0
- Support CIDv1/ipfs/bitswap/1.2.0
- Support Wantlist Have
's and Have
/DontHave
responsesBitswap implementations MUST support sending and receiving individual blocks of sizes less than or equal to 2MiB. Handling blocks larger than 2MiB is not recommended so as to keep compatibility with implementations which only support up to 2MiB.
Given that a client C wants to fetch data from some server S:
s_want
s_receive
. S MAY reuse this stream to send
back subsequent responses.
priority
values being responded to first.Cancel
message for that block to all peers from which it has not received a response
about that blockA single Bitswap message MAY contain any of the following content:
The wire format for Bitswap is simply a stream of Bitswap messages. The following protobuf describes the form of these messages. Note: all protobufs are described using proto3 syntax.
message Message {
message Wantlist {
message Entry {
bytes block = 1; // the block key, i.e. a CIDv0
int32 priority = 2; // the priority (normalized). default to 1
bool cancel = 3; // whether this revokes an entry
}
repeated Entry entries = 1; // a list of wantlist entries
bool full = 2; // whether this is the full wantlist. default to false
}
Wantlist wantlist = 1;
repeated bytes blocks = 2;
}
All protocol messages sent over a stream are prefixed with the message length in bytes, encoded as an unsigned variable length integer as defined by the multiformats unsigned-varint spec.
All protocol messages MUST be less than or equal to 4MiB in size.
Bitswap 1.1.0 introduces a payload
field to the protobuf message and deprecates the
existing 'blocks' field. The 'payload' field is an array of pairs of CID prefixes
and block data. The CID prefixes are used to ensure the correct codecs and hash
functions are used to handle the block on the receiving end.
It is otherwise identical to 1.0.0.
message Message {
message Entry {
bytes block = 1; // CID of the block
int32 priority = 2; // the priority (normalized). default to 1
bool cancel = 3; // whether this revokes an entry
}
repeated Entry entries = 1; // a list of wantlist entries
bool full = 2; // whether this is the full wantlist. default to false
}
message Block {
bytes prefix = 1; // CID prefix (all of the CID components except for the digest of the multihash)
bytes data = 2;
}
Wantlist wantlist = 1;
repeated Block payload = 3;
}
Bitswap 1.2.0 extends the Bitswap 1.1.0 protocol with the three changes:
Given that a client C wants to fetch data from some server S:
s_want
to S and sends a message for the blocks it wants:
Have
request) or for S to send the block (i.e. a Block
request).
C MAY also ask S to send back a DontHave
message in the event it doesn't
have the block.s_receive
. S MAY reuse this stream to send
back subsequent responses:
Have
request for data S has (and is willing to give
to C) it SHOULD respond with a Have
, although it MAY instead respond
with the block itself (e.g. if the block is very small).Have
request for data S does not have (or has but
is not willing to give to C) and C has requested for DontHave
responses
then S SHOULD respond with DontHave
.priority
values being responded to first.Cancel
message for that request to any peers that have not already responded
about that particular block. It SHOULD particularly send Cancel
messages for
Block
requests (as opposed to Have
requests) that have not yet been answered.message Message {
message Wantlist {
enum WantType {
Block = 0;
Have = 1;
}
message Entry {
bytes block = 1; // CID of the block
int32 priority = 2; // the priority (normalized). default to 1
bool cancel = 3; // whether this revokes an entry
WantType wantType = 4; // Note: defaults to enum 0, ie Block
bool sendDontHave = 5; // Note: defaults to false
}
repeated Entry entries = 1; // a list of wantlist entries
bool full = 2; // whether this is the full wantlist. default to false
}
message Block {
bytes prefix = 1; // CID prefix (all of the CID components except for the digest of the multihash)
bytes data = 2;
}
enum BlockPresenceType {
Have = 0;
DontHave = 1;
}
message BlockPresence {
bytes cid = 1;
BlockPresenceType type = 2;
}
Wantlist wantlist = 1;
repeated Block payload = 3;
repeated BlockPresence blockPresences = 4;
int32 pendingBytes = 5;
}
boxo/bitswap