Dealing with Encrypted Traffic

In this lesson, you will learn how to leverage TLS Server Name Indication (SNI) to enforce granular firewall policies without decrypting traffic.

When traffic is encrypted via TLS, the destination IP address and port are often insufficient for modern filtering. A single IP might host hundreds of websites via virtual hosting, making it impossible to block one domain while allowing another based solely on L3/L4 headers. Fortunately, the TLS handshake includes the SNI field—a plaintext header where the client explicitly specifies the hostname it wishes to reach.

Implementing SNI Filtering with NFTables

Because the SNI string appears in the ClientHello packet, we can use nftables to inspect the payload. By combining the ct (connection tracking) module with string matching, we can identify traffic directed at specific domains and apply rules accordingly.

Consider a scenario where you want to drop connections to a specific domain while allowing others on the same IP range:

# Define a set for blocked domains
set blocked_domains {
    type string
    flags interval
}

# Add a rule to inspect the payload of TCP packets
# Note: This checks the SNI header within the TLS handshake
nft add rule inet filter forward tcp dport 443 \
    @th,64,320 0x1603010000000000 \
    meta skb priority set 1 \
    ct helper set "tls_sni_helper"

nft add rule inet filter forward meta skb priority 1 \
    payload @th,320,800 string "blocked-domain.com" \
    drop

Challenges and Limitations

While powerful, this approach has specific constraints:

1. Performance Overhead: Deep packet inspection (DPI) at the kernel level for every flow introduces latency. Always ensure you are only inspecting the initial ClientHello packets.
2. Encrypted Client Hello (ECH): Modern TLS extensions like ECH aim to encrypt the SNI field itself. As these become standard, kernel-level SNI filtering will become less effective, eventually pushing filtering logic toward proxy-based solutions (which move beyond simple packet filtering).
3. Connection State: Since this relies on the ClientHello, ensure your connection tracking is correctly identifying the start of the TLS session; otherwise, the pattern match will fail to trigger.

By using these methods, you extend your firewall’s reach into the Application Layer while maintaining the performance benefits of native Linux kernel packet processing.

Key Takeaway: TLS SNI filtering allows you to enforce domain-specific policies on encrypted traffic by inspecting the plaintext handshake before the secure tunnel is fully established.