When the Host Has to Send a Packet, Packet is Thrown in Bucket

4 min readArticle

This is the sender-side view of the leaky bucket: when your application calls send(), the packet doesn't go straight onto the wire. It enters the bucket (transmit queue). The bucket decides when the packet actually goes out, based on the drain rate.

The Enqueue Process

shell

Application → write()/send() → Socket send buffer → NIC TX queue (bucket) → Wire

When the OS puts a packet into the transmit queue, three things can happen:

Queue has space: Packet is enqueued. It will be transmitted when the drain schedule reaches it.
Queue is full (bucket overflow): Packet is dropped at the sender. The application may get ENOBUFS or EAGAIN if non-blocking, or the kernel silently drops.
Rate limited by shaping qdisc: Packet is held in the shaping queue until the configured drain time arrives.

Fill Rate vs Drain Rate

The application controls fill rate. The network/shaping controls drain rate.

shell

Fill rate  (A): determined by how fast application calls send()
Drain rate (R): determined by link speed + any traffic shaping qdiscs

If A < R:   bucket is always nearly empty; packets go out immediately
If A = R:   steady state, small queue, minimal delay
If A > R:   queue grows; packets experience increasing latency; eventually overflow/drop

What Happens at the Application Level

When the bucket (socket send buffer) is full:

// Blocking socket (default): send() blocks until space is available
ssize_t sent = send(sockfd, buffer, len, 0);
// Application hangs here if send buffer is full

// Non-blocking socket: send() returns -1, errno = EAGAIN or EWOULDBLOCK
int flags = fcntl(sockfd, F_GETFL, 0);
fcntl(sockfd, F_SETFL, flags | O_NONBLOCK);
ssize_t sent = send(sockfd, buffer, len, 0);
if (sent == -1 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
    // bucket is full — back off, retry later
}

Socket Send Buffer = Application-Level Bucket

The socket send buffer is the application's view of the bucket:

bash

# Check current socket buffer sizes
sysctl net.core.wmem_default    # default send buffer
sysctl net.core.wmem_max        # max send buffer
sysctl net.ipv4.tcp_wmem        # TCP: min, default, max

# Increase send buffer for high-throughput applications
sudo sysctl -w net.core.wmem_max=16777216
sudo sysctl -w net.ipv4.tcp_wmem="4096 87380 16777216"

# Check a specific socket's send buffer size
ss -tm
# Output: skmem:(r0,rb87380,t0,tb87380,...) → tb = tx buffer size

The NIC TX Queue — The Physical Bucket

Below the socket buffer, the NIC has its own TX queue (ring buffer). This is the actual hardware-level bucket:

bash

# View NIC TX queue length
ip link show eth0
# -> qlen 1000   (default = 1000 packets)

# Check TX ring buffer size
ethtool -g eth0
# -> TX: 512   (hardware TX descriptor ring)

# View TX drops (overflow at NIC level)
ip -s link show eth0
# TX:  bytes  packets  errors  dropped  carrier  collisions
#      ...              0       47       0        0
# 'dropped' = TX queue overflow

# Increase TX queue length
sudo ip link set eth0 txqueuelen 5000

Practical Example: Watching Enqueue and Dequeue

bash

# Set a rate limit so we can observe the queueing
sudo tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 100ms

# Send a burst of traffic
ping -f -c 1000 <destination>   # flood ping, 1000 packets

# In another terminal, watch the queue depth
watch -n 0.1 "tc -s qdisc show dev eth0"
# Watch 'backlog' increase during burst, then drain at constant rate
# Watch 'dropped' increment if burst exceeds bucket (latency parameter)

# Clean up
sudo tc qdisc del dev eth0 root

The Overflow Decision — Drop or Block

When packets are thrown into a full bucket, the behavior depends on the layer:

Layer	Full bucket behavior
Application socket	`EAGAIN` (non-blocking) or `send()` blocks
OS network stack	Kernel drops, increments drop counter
tc qdisc	Configured action: drop / mark / delay
Hardware NIC ring	Hardware drops, increments `tx_dropped` counter

Impact on Application Design

For a security tool (scanner, packet injector, fuzzer), understanding the bucket model matters:

python

import socket
import time

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setblocking(False)

packets_sent = 0
packets_dropped = 0

for i in range(10000):
    try:
        s.sendto(b'X' * 1400, ('192.168.1.1', 9999))
        packets_sent += 1
    except BlockingIOError:
        packets_dropped += 1
        time.sleep(0.001)  # back off when bucket is full

print(f"Sent: {packets_sent}, Dropped at sender: {packets_dropped}")

The BlockingIOError (EAGAIN) tells you the send buffer bucket is full — your injection rate exceeds what the kernel can drain to the wire at that moment.

Reference

Linux kernel socket buffer internals: https://www.kernel.org/doc/html/latest/networking/
man 7 tcp — TCP socket options including SO_SNDBUF
man tc-tbf — traffic bucket filter
ethtool -g man page for NIC ring buffer tuning

techzonesite.comUnlock Your IT Potential