Yancy's Web Page

Welcome to my homepage!

      email at this domain
      yancy at irc.libera.chat

This site is Lynx (web browser) friendly (mostly).

20240114

      02/26/26

      As of late, I've been debugging a Bitcoin P2P application.  In order to better understand the
      communication that's happening under the hood, I resorted to running a tcpdump which captures
      packets between said application and Bitcoin core.  Note that my experimentation thus far is
      using the V1 protocol which is un-encrypted.

      To initiate the packet capture with tcpdump:
        sudo tcpdump -i `interface` -w `file` host `host ip`

      where:
        `host ip` is an instance of Bitcoin core
        `interface` is my wireless nic wlp3s0
        `file` is the location to save the packet capture (pcap file), like /tmp/dump

      Then, to read the pcap file: tcpdump -qns 0 -X -r `file`

      A Bitcoin P2P application begins communication with a handshake.  This handshake specifies the
      first message is a Version Message sent by the client. The Version Message is defined by the
      Protocol documentation.  According to the docs, all message structures, including version begin
      with some header information (see Message Structure in proto docs above).

      The Message structure memory layout is as follows:

      magic: 4 bytes
      command: 12 bytes
      length: 4 bytes
      checksum: 4 bytes
      payload: variable size

      In the case of Version, the "command" will be version in ASCII: 7665 7273 696f 6e00 0000 0000.
      So, back to the packet capture, to find where the communication between said application and
      Bitcoin core starts, I find the first occurrence of that ASCII.  On my capture, that occurs on
      the 6th packet midway in.  The next 4 bytes directly following the command is the length of
      the payload or message.  For my capture, this looks like `6200 0000`.  A quote from the
      Protocol docs is "Almost all integers are encoded in little endian", so when we see something
      like `6200 0000`, this tells me that the payload is 98 bytes long.  The remaining 4 bytes are
      the checksum which are un-important for now.

      b247 e642 - no clue what this is - 4 bytes
      7665 7273 696f 6e00 0000 0000 - this is the word version in hex - 12 bytes
      6200 0000 - this is the length of the payload (body) which is 98 bytes - 4 bytes
      ae4a f87f - this is the checksum - 4 bytes

      As a side note, the 4 bytes directly proceeding the command should be the network magic.
      However, `b247 e642` are the bytes directly proceeding version, and that does not translate
      to any know "magic values" in the docs.

      Continuing on to the payload, AKA message body, which is of variable length, however in this
      case is 98 bytes.

      The payload, which is of type Version Message is as follows:

      version: 4 bytes
      services: 8 bytes
      timestamp: 8 bytes
      receive address: 26 bytes
      from address: 26 bytes
      nonce: 8 bytes
      user agent: variable size
      start height: 4 bytes
      realy: 1 bytes

      I find the next 4 bytes directly following checksum to be `7f11 0100` which translates to
      70015 (little endian).

      The payload continues as follows:

      7f11 0100 - this is the protocol version, which in decimal is 70015 (little endian) - 4 bytes
      0000 0000 0000 0000 - services - 8 bytes
      a83f 9c69 0000 0000 - timestamp - 8 bytes
      0000 0000 0000 0000 0000 0000 0000 0000 0000 ffff 7f00 0001 d431 - address rcv - 26 bytes
      0000 0000 0000 0000 0000 0000 0000 0000 0000 ffff 7f00 0001 d431 - address send - 26 bytes
      0000 0000 0000 0000 - nonce - 4 bytes
      0c2f 6c6f 7265 3a30 2e30 2e31 2f - user agent - ?
      0000 0000 - start height - 4 bytes
      00 - relay - 1

      User agent is a field to transmit the name of the software, whether it be Bitcoin Core or some
      other client.  Technically the field is of variable length, however since we know the length
      of the payload, 98 bytes, and all other fields are of known size, we can also know the length
      of user agent with some simple arithmetic: 98 - 4 - 8 - 8 - 26 - 26 - 8 - 4 - 1.  Also notice
      that the sending and the receiving address are the same.  This is no doubt because this is
      the first communication, so there is only really a sending address.  It's curious though that
      the receiving address would not simply be empty.

      The response from Bitcoin core will be the version message, so I look for the next occurrence
      of "version" in ASCII 7665 7273 696f 6e00 0000 0000.  Using the version hex as an anchor, I look at
      the proceeding and following bytes which reveal the following:

      magic: fabf b5da
      command: 7665 7273 696f 6e00 0000 0000
      length: 6700 0000
      checksum 4f01 cf12
      payload: ?

      This part threw me for a loop though.  I was expecting to find the payload directly following
      checksum, which is how the first version message by the initiating client appeared in the pcap
      file.  However, for whatever reason, bitcoin core sends the header and the message body in
      two separate packets.  There must be some bytes that go into the beginning of the header
      which causes the pcap file to have a payload _which does not directly follow_ the checksum.
      Looking halfway down the packet that follows Bitcoin Cores first packet response, I find:

      version: 8011 0100 - protocol version 70016
      services: 090c 0000
      timestamp: 0000 0000
      addr recv: 533f 9c69 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
      addr from: 0000 0000 0000 0000 090c 0000 0000 0000 0000 0000 0000 0000 0000
      nonce: 0000 0000
      user agent: 0000 0000 43dc 33f0  6041 cf80 112f 5361 746f 7368 693a 3330 2e39 392e 302f
      start height: 0000 0000
      relay: 01

03/07/26

Continuing on from last post on 02/26, in which I detailed the start of the handshake
between two peers. After the version message is first sent from initiating client,
the receiving peer, a core node, responds sending back it's version message. The core node
now knows both the sender and it's own capable protocol version, so it computes the minimum
of the two versions. In the trace above, my initial request set protocol version `70015`
and the receiver, in this case a core node, responded with `70016`. Therefore, on the core
node side, the session protocol will be computed as `70015`.

In response to the version message, the core node sends a `verack` message, a header only
message without a body. You might say, a disembodied message.

fabf b5da - network magic
7665 7261 636b 0000 0000 0000 - verack command
0000 0000 - note the length is zero signifying no body
5df6 e0e2 - checksum

The initiating client will do the same in response to the version message and set the
session protocol for communication going forward.

L -> R: Send version message with the local peer's version
R -> L: Send version message back
R -> L: Send verack message
R: Sets version to the minimum of the 2 versions
L -> R: Send verack message after receiving version message from R
L: Sets version to the minimum of the 2 versions
[0]

Now that we know the session protocol version `70015` AKA:
`ProtocolVersion::INVALID_CB_NO_BAN_VERSION` in rust-bitcoin land [1], we know what
messages to see next in our trace.

The next message in the trace is a `sendcmpct` which is sent from core to the initiating
client.

Here is the `sendcmpct` header:
fabf b5da - magic
7365 6e64 636d 7063 7400 0000 - verack cmd
0900 0000 - length 9
e92f 5ef8 - checksum

Tcpdump shows the size of the contained data in the next packet as `9` which matches what
we just saw in the header (length 9).

Body (payload):
00 - mode (low-bandwidth)
02 0000 0000 0000 00 - pch command

Further discussion of `sendcmpct` can be found here [2].

The very next message received from core is a ping command. Ping [3] is used in protocol
versions greater than `60000`, and since we negotiated a session version of `70015`,
needing to respond to ping is no surprise.

Ping Header:
fabf b5da - magic
7069 6e67 0000 0000 0000 0000 - ping command
0800 0000 - length
667e 2273 - checksum

Ping Body:
432f 5f84 912b c7de - nonce

The client which initiated the communication in the first place with core, ought now respond
with a Pong message using the corresponding nonce.

0. version handshake
1. rust-bitcoin
2. bip-0152
2. bip-0031