quic 協議及核心源碼分析
quic 協議
1、網絡通信時，爲了確保數據不丟包，早在幾十年前就發明了 tcp 協議！然而此一時非彼一時，隨着技術進步和業務需求增多，tcp 也暴露了部分比較明顯的缺陷，比如:
-
建立連接的 3 次握手延遲大； TLS 需要至少需要 2 個 RTT，延遲也大
-
協議缺陷可能導致 syn 反射類的 DDOS 攻擊
-
tcp 協議緊耦合到了操作系統，升級需要操作系統層面改動，無法快速、大面積推廣升級補丁包
-
對頭阻塞：數據被分成 sequence，一旦中間的 sequence 丟包，後面的 sequence 也不會處理
-
中轉設備僵化：路由器、交換機等設備 “認死理”，比如只認 80、443 等端口，其他端口一律丟棄
爲了解決這些問題，牛逼 plus 的 google 早在 10 年前，也就是 2012 年發佈了基於 UDP 的 quic 協議！爲啥不基於 tcp 了，因爲 tcp 有上述 5 條缺陷的嘛，所以乾脆 “另起爐竈” 重新開搞！
2、正式介紹前，先看一張圖：quci 在右邊，底層用了 udp 的協議；自生實現了 Multistreaming、tls、擁塞控制，然後支撐了上層的 http/2，所以我個人理解 quic 是一個夾在應用層和傳輸層之間的協議！
上面 “數落” 了 tcp 協議的 5 點不是，quic 又是怎麼基於 udp 解決這些問題的了？quic 是基於 UDP 實現的協議，而 UDP 是不可靠的面向報文的協議，這和 TCP 基於 IP 層的實現並沒有什麼本質上的不同，都是：
-
底層只負責盡力而爲的，以 packet 爲單位的傳輸;
-
上層協議實現更關鍵的特性，如可靠，有序，安全等。
（1）由於 quic 並未改造 udp，而是直接使用 udp，所以不需要改動現有的操作系統，也兼容了現有的網絡中轉設備，這些都不需要做任何改動，所以 quic 部署的改造成本相對較低！但是 quic 畢竟是新的協議，在哪部署和使用了？只有應用層了！這個和操作系統是解耦的，全靠 3 環的 app 自己想辦法實現（和之前介紹的協程是不是類似了？）！google 已經開源了算法，下載連接見文章末尾的參考 5；PS：微軟也實現了 QUIC 協議，名稱叫 MsQuic，源碼在這：https://github.com/microsoft/msquic；
這裏多說幾句：應用層 app 能操作的最底層協議就是傳輸層了。大家在用 libc 庫編寫通信代碼時可以對指定的 ip 地址和端口收發數據，沒法改自己的 mac 地址吧？也沒法改自己的 ip 地址吧？這些都是操作系統內核封裝的，app 的開發人員是不需要、也是沒法改變的，所以站在安全防護的角度，部分大廠基於傳輸層自研了類似 quic 的通信協議，逆向時需要人工挨個分析協議字段的含義了，現成的 fiddler/charles/burpsuit 等 https/http 的抓包工具是無效的，用 wireshark 這類工具抓包也無法自動解析這些廠家自研的協議！
（2）TCP 連接需要 3 次握手，tls 最少需要 2 次 RTT，兩個加起來一共要耗費 5 個 RTT，究其原因一方面是 TCP 和 TLS 分層設計導致的：分層的設計需要每個邏輯層次分別建立自己的連接狀態。另一方面是 TLS 的握手階段複雜的密鑰協商機制導致的，quic 又是怎麼改進的了？quic 建立握手的步驟如下：
-
客戶端判斷本地是否已有服務器的全部配置參數（證書配置信息），如果有則直接跳轉到 (5)，否則繼續 。
-
客戶端向服務器發送 inchoate client hello(CHLO) 消息，請求服務器傳輸配置參數。
-
服務器收到 CHLO，回覆 rejection(REJ) 消息，其中包含服務器的部分配置參數
-
客戶端收到 REJ，提取並存儲服務器配置參數，跳回到 (1) 。
-
客戶端向服務器發送 full client hello 消息，開始正式握手，消息中包括客戶端選擇的公開數。此時客戶端根據獲取的服務器配置參數和自己選擇的公開數，可以計算出初始密鑰 K1。
-
服務器收到 full client hello，如果不同意連接就回復 REJ，同 (3)；如果同意連接，根據客戶端的公開數計算出初始密鑰 K1，回覆 server hello(SHLO) 消息， SHLO 用初始密鑰 K1 加密，並且其中包含服務器選擇的一個臨時公開數。
-
客戶端收到服務器的回覆，如果是 REJ 則情況同 (4)；如果是 SHLO，則嘗試用初始密鑰 K1 解密，提取出臨時公開數。
-
客戶端和服務器根據臨時公開數和初始密鑰 K1，各自基於 SHA-256 算法推導出會話密鑰 K2。
-
雙方更換爲使用會話密鑰 K2 通信，初始密鑰 K1 此時已無用，QUIC 握手過程完畢。之後會話密鑰 K2 更新的流程與以上過程類似，只是數據包中的某些字段略有不同。這裏爲啥不繼續使用 key1，而是要重新生成 key2 來加密了？核心是爲了前向安全！萬一 key1 泄漏，之前用 key1 加密的數據全都被解密。所以爲了前向安全，每次通信時會重新生成 key2 加密！
總的來說：
-
udp 本身就不是面向連接的協議，所以省略了 tcp 3 次握手連接的耗時；直接通過事先內置的服務器參數發起通信請求；
-
既然不是面向連接的，怎麼確保所有的數據都能到達了？通過 stream id 和 stream offset 確保數據包不會丟失，接收方能收到完整的全量數據
-
第一次用 DH 算法計算對稱加密的密鑰需要 1 個 RTT；後續每次都用這個緩存的密鑰加密，又省了一個 RTT；本質上是把 tcp 的打招呼、握手，還有 tls 交換密鑰的工作在 1 個 RTT 中全做了，這就是相比於 tcp 實現的 tls 效率高的根本原因！
注意：通信雙方用於密鑰交換的 DH 算法無法防止中間人攻擊，所以僅通過密鑰交換是無法防止被抓包的，所以還要通過證書等其他方式驗證身份！x 音就是通過 libboringssl.so（google 開源的一個 openssl 分支）SSL_CTX_set_custom_verify 函數驗證客戶端是否是原來的 client，而不是抓包軟件！
（3）擁塞控制：QUIC 使用可插拔的擁塞控制，相較於 TCP，它能提供更豐富的擁塞控制信息。比如對於每一個包，不管是原始包還是重傳包，都帶有一個新的序列號 (seq)，這使得 QUIC 能夠區分 ACK 是重傳包還是原始包，從而避免了 TCP 重傳模糊的問題。QUIC 同時還帶有收到數據包與發出 ACK 之間的時延信息。這些信息能夠幫助更精確的計算 RTT！同時，因爲 quic 不依賴操作系統，而是在應用層實現，所以開發人員對於 quic 有非常強的操控能力：完全可以根據不同的業務場景，實現和配置不同的擁塞控制算法以及參數；比如 Google 提出的 BBR 擁塞控制算法與 CUBIC 是思路完全不一樣的算法，在弱網和一定丟包場景，BBR 比 CUBIC 更不敏感，性能也更好；
（4）隊頭阻塞：TCP 爲了保證可靠性，使用了基於字節序號的 Sequence Number 及 Ack 來確認消息的有序到達；一旦中間某個 sequence 的包丟失，哪怕是這個 sequence 後面的數據已經到達接收端，操作系統也不會立即把數據發給上層的應用來接受處理，而是一直等待發送端重新發送丟失的 sequence 包，舉例如下：
應用層可以順利讀取 stream1 中的內容，但由於 stream2 中的第三個 segment 發生了丟包，TCP 爲了保證數據的可靠性，需要發送端重傳第 3 個 segment 才能通知應用層讀取接下去的數據。所以即使 stream3、stream4 的內容已順利抵達，應用層仍然無法讀取，只能等待 stream2 中丟失的包進行重傳。在弱網環境下，HTTP2 的隊頭阻塞問題在用戶體驗上極爲糟糕！quic 是怎麼既確保數據傳輸可靠不丟失，又解決隊頭阻塞的這個問題的了？
對於數據包的傳輸，肯定是要編號的，否則接受方在拼接這些數據包的時候怎麼知道順序了？quic 協議用 Packet Number 代替了 TCP 的 Sequence Number，不同之處在於：
-
每個 Packet Number 都嚴格遞增，也就是說就算 Packet N 丟失了，重傳的 Packet N 的 Packet Number 已經不是 N，而是一個比 N 大的值，比如 Packet N+M；
-
數據包支持亂序確認，不再要求 TCP 那樣必須有序確認
當數據包 Packet N 丟失後，只要有新的已接收數據包確認，當前窗口就會繼續向右滑動。待發送端獲知數據包 Packet N 丟失後，會將需要重傳的數據包放到待發送隊列，重新編號比如數據包 Packet N+M 後重新發送給接收端，對重傳數據包的處理跟發送新的數據包類似，這樣就不會因爲丟包重傳將當前窗口阻塞在原地，從而解決了隊頭阻塞問題；但是問題又來了：怎麼確認 Package N+M 就是重傳 PackageN 的數據包了？這就涉及到 quic 另一個重要的特性了：多路複用！比如用戶訪問某個網頁，這個頁面有兩個文件，分別是 index.htm 和 index.js，可以同時、分別傳輸這兩個文件！每個傳輸的 stream 都有各自的 id，所以可以通過 id 確認是哪個 stream 超時丟包了！但包的 Packet 編號是 N+M，怎麼進一步確認就是重傳的 Packet N 包了？這就需要另一個重要的變量了：offset！怎麼樣，單從英語是不是就能猜到這個變量的作用了？每個數據包都有個 offset 字段，用於標識在 stream id 中的偏移！接收方完全可以根據 offset 來拼接收到的數據包！
總結：quic 協議可以在亂序發送的情況下任然可靠不丟失，靠的就是每個數據包的 offset 字段；再搭配上 stream id 字段，接收方完全可以在亂序的情況下無誤拼接收到的數據包了！
（4）除了以上通過 stream id 和 stream offset 確保數據不丟失外，quic 還採用了另一個叫向前糾錯 (Forward Error Correction，FEC) 的校驗方式：即每個數據包除了它本身的內容之外，還包括了部分其他數據包的數據，因此少量的丟包可以通過其他包的冗餘數據直接組裝而無需重傳。向前糾錯犧牲了每個數據包可以發送數據的上限，但是減少了因爲丟包導致的數據重傳，因爲數據重傳將會消耗更多的時間 (包括確認數據包丟失、請求重傳、等待新數據包等步驟的時間消耗)；這個原理和糾刪碼沒有本質區別！
（5）通信雙方不論使用何種協議，發送的數據必須事前約定好格式，否則接受方怎麼從數據包（本質就是一段字符串）中解析和提取關鍵的信息了？quic 協議的格式如下：
數據包中除了個別報文比如 PUBLIC_RESET 和 CHLO，所有報文頭部（上圖紅色部分）都是經過認證的（哈希散列值），報文 Body （上圖綠色部分）都是經過加密的，這樣只要對 QUIC 報文任何修改，接收端都能夠及時發現；每個字段的含義如下：
-
Flags：用於表示 Connection ID 長度、Packet Number 長度等信息；
-
Connection ID：客戶端隨機選擇的最大長度爲 64 位的無符號整數，用於標識連接；如果 app 更換了 ip 地址（比如 wifi 和 4G 之間切換了），仍然可以通過這個 id 和服務端在 0 RTT 下通信！
-
QUIC Version：QUIC 協議的版本號，32 位的可選字段。如果 Public Flag & FLAG_VERSION != 0，這個字段必填。客戶端設置 Public Flag 中的 Bit0 爲 1，並且填寫期望的版本號。如果客戶端期望的版本號服務端不支持，服務端設置 Public Flag 中的 Bit0 爲 1，並且在該字段中列出服務端支持的協議版本（0 或者多個），並且該字段後不能有任何報文；
-
Packet Number：長度取決於 Public Flag 中 Bit4 及 Bit5 兩位的值，最大長度 6 字節。發送端在每個普通報文中設置 Packet Number。發送端發送的第一個包的序列號是 1，隨後的數據包中的序列號的都大於前一個包中的序列號；
-
Stream ID：用於標識當前數據流屬於哪個資源請求，用於消除隊頭阻塞；
-
Offset：標識當前數據包在當前 Stream ID 中的字節偏移量，用於消除隊頭阻塞。
（6）爲了便於理解和記憶，這裏把 quic 的要點做了總結，如下：
3、正式因爲 quic 有這麼多優點，國內很多互聯網一、二線廠商都開始採用，其中比較著名的 app 就是 x 音了！lib 庫中有個 libsscronet.so 就支持 quic 協議！
quic 協議核心源碼
quic 協議最早是 google 提出來的，所以狗家的源碼肯定是最 “正宗” 的！
1、quic 相比 tcp 實現的 tls，前面省略了 3~4 個 RTT，根因就是發起連接請求時就發送自己的公鑰給對方，讓對方利用自己的公鑰計算後續對稱加密的 key，這就是所謂的 handshake；在 libquic-master\src\net\quic\core\quic_crypto_client_stream.cc 中有具體實現握手的代碼，先看 DoHandshakeLoop 函數：
void QuicCryptoClientStream::DoHandshakeLoop(const CryptoHandshakeMessage* in) {
QuicCryptoClientConfig::CachedState* cached =
crypto_config_->LookupOrCreate(server_id_);
QuicAsyncStatus rv = QUIC_SUCCESS;
do {
CHECK_NE(STATE_NONE, next_state_);
const State state = next_state_;
next_state_ = STATE_IDLE;
rv = QUIC_SUCCESS;
switch (state) {
case STATE_INITIALIZE:
DoInitialize(cached);
break;
case STATE_SEND_CHLO:
DoSendCHLO(cached);
return; // return waiting to hear from server.
case STATE_RECV_REJ:
DoReceiveREJ(in, cached);
break;
case STATE_VERIFY_PROOF:
rv = DoVerifyProof(cached);
break;
case STATE_VERIFY_PROOF_COMPLETE:
DoVerifyProofComplete(cached);
break;
case STATE_GET_CHANNEL_ID:
rv = DoGetChannelID(cached);
break;
case STATE_GET_CHANNEL_ID_COMPLETE:
DoGetChannelIDComplete();
break;
case STATE_RECV_SHLO:
DoReceiveSHLO(in, cached);
break;
case STATE_IDLE:
// This means that the peer sent us a message that we weren't expecting.
CloseConnectionWithDetails(QUIC_INVALID_CRYPTO_MESSAGE_TYPE,
"Handshake in idle state");
return;
case STATE_INITIALIZE_SCUP:
DoInitializeServerConfigUpdate(cached);
break;
case STATE_NONE:
NOTREACHED();
return; // We are done.
}
} while (rv != QUIC_PENDING && next_state_ != STATE_NONE);
}只要 quic 的狀態不是 pending，並且下一個狀態不是 NONE，就根據不同的狀態調用不同的處理函數！具體發送 handshake 小的函數是 DoSendCHLO，代碼如下：
/*發送client hello消息*/
void QuicCryptoClientStream::DoSendCHLO(
QuicCryptoClientConfig::CachedState* cached) {
if (stateless_reject_received_) {//如果收到了server拒絕的消息
// If we've gotten to this point, we've sent at least one hello
// and received a stateless reject in response. We cannot
// continue to send hellos because the server has abandoned state
// for this connection. Abandon further handshakes.
next_state_ = STATE_NONE;
if (session()->connection()->connected()) {
session()->connection()->CloseConnection(//關閉連接
QUIC_CRYPTO_HANDSHAKE_STATELESS_REJECT, "stateless reject received",
ConnectionCloseBehavior::SILENT_CLOSE);
}
return;
}
// Send the client hello in plaintext.
//注意：這是client hello消息，沒必要加密
session()->connection()->SetDefaultEncryptionLevel(ENCRYPTION_NONE);
encryption_established_ = false;
if (num_client_hellos_ > kMaxClientHellos) {//握手消息已經發送了很多，不能再發了
CloseConnectionWithDetails(
QUIC_CRYPTO_TOO_MANY_REJECTS,
base::StringPrintf("More than %u rejects", kMaxClientHellos).c_str());
return;
}
num_client_hellos_++;
//開始構造握手消息了
CryptoHandshakeMessage out;
DCHECK(session() != nullptr);
DCHECK(session()->config() != nullptr);
// Send all the options, regardless of whether we're sending an
// inchoate or subsequent hello.
/*填充握手消息的各個字段*/
session()->config()->ToHandshakeMessage(&out);
// Send a local timestamp to the server.
out.SetValue(kCTIM,
session()->connection()->clock()->WallNow().ToUNIXSeconds());
if (!cached->IsComplete(session()->connection()->clock()->WallNow())) {
crypto_config_->FillInchoateClientHello(
server_id_, session()->connection()->supported_versions().front(),
cached, session()->connection()->random_generator(),
/* demand_x509_proof= */ true, &crypto_negotiated_params_, &out);
// Pad the inchoate client hello to fill up a packet.
const QuicByteCount kFramingOverhead = 50; // A rough estimate.
const QuicByteCount max_packet_size =
session()->connection()->max_packet_length();
if (max_packet_size <= kFramingOverhead) {
DLOG(DFATAL) << "max_packet_length (" << max_packet_size
<< ") has no room for framing overhead.";
CloseConnectionWithDetails(QUIC_INTERNAL_ERROR,
"max_packet_size too smalll");
return;
}
if (kClientHelloMinimumSize > max_packet_size - kFramingOverhead) {
DLOG(DFATAL) << "Client hello won't fit in a single packet.";
CloseConnectionWithDetails(QUIC_INTERNAL_ERROR, "CHLO too large");
return;
}
// TODO(rch): Remove this when we remove:
// FLAGS_quic_use_chlo_packet_size
out.set_minimum_size(
static_cast<size_t>(max_packet_size - kFramingOverhead));
next_state_ = STATE_RECV_REJ;
/*做hash簽名，接收方會根據hash驗證消息是否完整*/
CryptoUtils::HashHandshakeMessage(out, &chlo_hash_);
//發送消息
SendHandshakeMessage(out);
return;
}
// If the server nonce is empty, copy over the server nonce from a previous
// SREJ, if there is one.
if (FLAGS_enable_quic_stateless_reject_support &&
crypto_negotiated_params_.server_nonce.empty() &&
cached->has_server_nonce()) {
crypto_negotiated_params_.server_nonce = cached->GetNextServerNonce();
DCHECK(!crypto_negotiated_params_.server_nonce.empty());
}
string error_details;
/*繼續填充client hello消息*/
QuicErrorCode error = crypto_config_->FillClientHello(
server_id_, session()->connection()->connection_id(),
session()->connection()->version(),
session()->connection()->supported_versions().front(), cached,
session()->connection()->clock()->WallNow(),
//這個隨機數會被server用來計算生成對稱加密的key
session()->connection()->random_generator(),
channel_id_key_.get(),
//保存了nonce、key、token相關信息；後續對稱加密的方法是CTR，需要NONCE值
&crypto_negotiated_params_,
&out, &error_details);
if (error != QUIC_NO_ERROR) {
// Flush the cached config so that, if it's bad, the server has a
// chance to send us another in the future.
cached->InvalidateServerConfig();
CloseConnectionWithDetails(error, error_details);
return;
}
/*繼續對消息做hash，便於server驗證收到的消息是否完整*/
CryptoUtils::HashHandshakeMessage(out, &chlo_hash_);
channel_id_sent_ = (channel_id_key_.get() != nullptr);
if (cached->proof_verify_details()) {
proof_handler_->OnProofVerifyDetailsAvailable(
*cached->proof_verify_details());
}
next_state_ = STATE_RECV_SHLO;
SendHandshakeMessage(out);
// Be prepared to decrypt with the new server write key.
session()->connection()->SetAlternativeDecrypter(
ENCRYPTION_INITIAL,
crypto_negotiated_params_.initial_crypters.decrypter.release(),
true /* latch once used */);
// Send subsequent packets under encryption on the assumption that the
// server will accept the handshake.
session()->connection()->SetEncrypter(
ENCRYPTION_INITIAL,
crypto_negotiated_params_.initial_crypters.encrypter.release());
session()->connection()->SetDefaultEncryptionLevel(ENCRYPTION_INITIAL);
// TODO(ianswett): Merge ENCRYPTION_REESTABLISHED and
// ENCRYPTION_FIRST_ESTABLSIHED
encryption_established_ = true;
session()->OnCryptoHandshakeEvent(QuicSession::ENCRYPTION_REESTABLISHED);
}個人覺得最核心的代碼就是 FillClientHello 函數了，這裏會生成隨機數，後續 server 會利用這個隨機數生成對稱加密的 key！部分通信的參數也會通過這個函數的執行保存在 crypto_negotiated_params_對象中！client 發送了 hello 包，接下來該 server 處理這個包了，代碼在 libquic-master\src\net\quic\core\quic_crypto_server_stream.cc 和 quic_crypto_server_config.cc 中，代碼如下：核心功能是生成自己的公鑰，還有後續對稱加密的 key！
QuicErrorCode QuicCryptoServerConfig::ProcessClientHello(
const ValidateClientHelloResultCallback::Result& validate_chlo_result,
bool reject_only,
QuicConnectionId connection_id,
const IPAddress& server_ip,
const IPEndPoint& client_address,
QuicVersion version,
const QuicVersionVector& supported_versions,
bool use_stateless_rejects,
QuicConnectionId server_designated_connection_id,
const QuicClock* clock,
QuicRandom* rand,//發送給client用於計算對稱key
QuicCompressedCertsCache* compressed_certs_cache,
QuicCryptoNegotiatedParameters* params,
QuicCryptoProof* crypto_proof,
QuicByteCount total_framing_overhead,
QuicByteCount chlo_packet_size,
CryptoHandshakeMessage* out,
DiversificationNonce* out_diversification_nonce,
string* error_details) const {
DCHECK(error_details);
const CryptoHandshakeMessage& client_hello =
validate_chlo_result.client_hello;
const ClientHelloInfo& info = validate_chlo_result.info;
QuicErrorCode valid = CryptoUtils::ValidateClientHello(
client_hello, version, supported_versions, error_details);
if (valid != QUIC_NO_ERROR)
return valid;
StringPiece requested_scid;
client_hello.GetStringPiece(kSCID, &requested_scid);
const QuicWallTime now(clock->WallNow());
scoped_refptr<Config> requested_config;
scoped_refptr<Config> primary_config;
{
base::AutoLock locked(configs_lock_);
if (!primary_config_.get()) {
*error_details = "No configurations loaded";
return QUIC_CRYPTO_INTERNAL_ERROR;
}
if (!next_config_promotion_time_.IsZero() &&
next_config_promotion_time_.IsAfter(now)) {
SelectNewPrimaryConfig(now);
DCHECK(primary_config_.get());
DCHECK_EQ(configs_.find(primary_config_->id)->second, primary_config_);
}
// Use the config that the client requested in order to do key-agreement.
// Otherwise give it a copy of |primary_config_| to use.
primary_config = crypto_proof->config;
requested_config = GetConfigWithScid(requested_scid);
}
if (validate_chlo_result.error_code != QUIC_NO_ERROR) {
*error_details = validate_chlo_result.error_details;
return validate_chlo_result.error_code;
}
out->Clear();
if (!ClientDemandsX509Proof(client_hello)) {
*error_details = "Missing or invalid PDMD";
return QUIC_UNSUPPORTED_PROOF_DEMAND;
}
DCHECK(proof_source_.get());
string chlo_hash;
CryptoUtils::HashHandshakeMessage(client_hello, &chlo_hash);
// No need to get a new proof if one was already generated.
if (!crypto_proof->chain &&
!proof_source_->GetProof(server_ip, info.sni.as_string(),
primary_config->serialized, version, chlo_hash,
&crypto_proof->chain, &crypto_proof->signature,
&crypto_proof->cert_sct)) {
return QUIC_HANDSHAKE_FAILED;
}
StringPiece cert_sct;
if (client_hello.GetStringPiece(kCertificateSCTTag, &cert_sct) &&
cert_sct.empty()) {
params->sct_supported_by_client = true;
}
if (!info.reject_reasons.empty() || !requested_config.get()) {
BuildRejection(version, clock->WallNow(), *primary_config, client_hello,
info, validate_chlo_result.cached_network_params,
use_stateless_rejects, server_designated_connection_id, rand,
compressed_certs_cache, params, *crypto_proof,
total_framing_overhead, chlo_packet_size, out);
return QUIC_NO_ERROR;
}
if (reject_only) {
return QUIC_NO_ERROR;
}
const QuicTag* their_aeads;
const QuicTag* their_key_exchanges;
size_t num_their_aeads, num_their_key_exchanges;
if (client_hello.GetTaglist(kAEAD, &their_aeads, &num_their_aeads) !=
QUIC_NO_ERROR ||
client_hello.GetTaglist(kKEXS, &their_key_exchanges,
&num_their_key_exchanges) != QUIC_NO_ERROR ||
num_their_aeads != 1 || num_their_key_exchanges != 1) {
*error_details = "Missing or invalid AEAD or KEXS";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
size_t key_exchange_index;
if (!QuicUtils::FindMutualTag(requested_config->aead, their_aeads,
num_their_aeads, QuicUtils::LOCAL_PRIORITY,
¶ms->aead, nullptr) ||
!QuicUtils::FindMutualTag(requested_config->kexs, their_key_exchanges,
num_their_key_exchanges,
QuicUtils::LOCAL_PRIORITY,
¶ms->key_exchange, &key_exchange_index)) {
*error_details = "Unsupported AEAD or KEXS";
return QUIC_CRYPTO_NO_SUPPORT;
}
if (!requested_config->tb_key_params.empty()) {
const QuicTag* their_tbkps;
size_t num_their_tbkps;
switch (client_hello.GetTaglist(kTBKP, &their_tbkps, &num_their_tbkps)) {
case QUIC_CRYPTO_MESSAGE_PARAMETER_NOT_FOUND:
break;
case QUIC_NO_ERROR:
if (QuicUtils::FindMutualTag(
requested_config->tb_key_params, their_tbkps, num_their_tbkps,
QuicUtils::LOCAL_PRIORITY, ¶ms->token_binding_key_param,
nullptr)) {
break;
}
default:
*error_details = "Invalid Token Binding key parameter";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
}
StringPiece public_value;
/*提取client hello數據包發送的公鑰，server要用來生成對稱加密的key*/
if (!client_hello.GetStringPiece(kPUBS, &public_value)) {
*error_details = "Missing public value";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
const KeyExchange* key_exchange =
requested_config->key_exchanges[key_exchange_index];
if (!key_exchange->CalculateSharedKey(public_value,
¶ms->initial_premaster_secret)) {
*error_details = "Invalid public value";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
if (!info.sni.empty()) {
std::unique_ptr<char[]> sni_tmp(new char[info.sni.length() + 1]);
memcpy(sni_tmp.get(), info.sni.data(), info.sni.length());
sni_tmp[info.sni.length()] = 0;
params->sni = CryptoUtils::NormalizeHostname(sni_tmp.get());
}
string hkdf_suffix;
//client hello消息序列化，便於提取？
const QuicData& client_hello_serialized = client_hello.GetSerialized();
/*根據一個原始密鑰材料，用hkdf算法推導出指定長度的密鑰；
這裏明顯是要根據client hello的數據生成對稱加密的密鑰了
*/
hkdf_suffix.reserve(sizeof(connection_id) + client_hello_serialized.length() +
requested_config->serialized.size());
hkdf_suffix.append(reinterpret_cast<char*>(&connection_id),
sizeof(connection_id));
hkdf_suffix.append(client_hello_serialized.data(),
client_hello_serialized.length());
hkdf_suffix.append(requested_config->serialized);
DCHECK(proof_source_.get());
if (crypto_proof->chain->certs.empty()) {
*error_details = "Failed to get certs";
return QUIC_CRYPTO_INTERNAL_ERROR;
}
hkdf_suffix.append(crypto_proof->chain->certs.at(0));
StringPiece cetv_ciphertext;
if (requested_config->channel_id_enabled &&
client_hello.GetStringPiece(kCETV, &cetv_ciphertext)) {
CryptoHandshakeMessage client_hello_copy(client_hello);
client_hello_copy.Erase(kCETV);
client_hello_copy.Erase(kPAD);
const QuicData& client_hello_copy_serialized =
client_hello_copy.GetSerialized();
string hkdf_input;
hkdf_input.append(QuicCryptoConfig::kCETVLabel,
strlen(QuicCryptoConfig::kCETVLabel) + 1);
hkdf_input.append(reinterpret_cast<char*>(&connection_id),
sizeof(connection_id));
hkdf_input.append(client_hello_copy_serialized.data(),
client_hello_copy_serialized.length());
hkdf_input.append(requested_config->serialized);
CrypterPair crypters;
if (!CryptoUtils::DeriveKeys(params->initial_premaster_secret, params->aead,
info.client_nonce, info.server_nonce,
hkdf_input, Perspective::IS_SERVER,
CryptoUtils::Diversification::Never(),
&crypters, nullptr /* subkey secret */)) {
*error_details = "Symmetric key setup failed";
return QUIC_CRYPTO_SYMMETRIC_KEY_SETUP_FAILED;
}
char plaintext[kMaxPacketSize];
size_t plaintext_length = 0;
const bool success = crypters.decrypter->DecryptPacket(
kDefaultPathId, 0 /* packet number */,
StringPiece() /* associated data */, cetv_ciphertext, plaintext,
&plaintext_length, kMaxPacketSize);
if (!success) {
*error_details = "CETV decryption failure";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
std::unique_ptr<CryptoHandshakeMessage> cetv(
CryptoFramer::ParseMessage(StringPiece(plaintext, plaintext_length)));
if (!cetv.get()) {
*error_details = "CETV parse error";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
StringPiece key, signature;
if (cetv->GetStringPiece(kCIDK, &key) &&
cetv->GetStringPiece(kCIDS, &signature)) {
if (!ChannelIDVerifier::Verify(key, hkdf_input, signature)) {
*error_details = "ChannelID signature failure";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
params->channel_id = key.as_string();
}
}
string hkdf_input;
size_t label_len = strlen(QuicCryptoConfig::kInitialLabel) + 1;
hkdf_input.reserve(label_len + hkdf_suffix.size());
hkdf_input.append(QuicCryptoConfig::kInitialLabel, label_len);
hkdf_input.append(hkdf_suffix);
string* subkey_secret = ¶ms->initial_subkey_secret;
CryptoUtils::Diversification diversification =
CryptoUtils::Diversification::Never();
if (version > QUIC_VERSION_32) {
rand->RandBytes(out_diversification_nonce->data(),
out_diversification_nonce->size());
diversification =
CryptoUtils::Diversification::Now(out_diversification_nonce);
}
if (!CryptoUtils::DeriveKeys(params->initial_premaster_secret, params->aead,
info.client_nonce, info.server_nonce, hkdf_input,
Perspective::IS_SERVER, diversification,
¶ms->initial_crypters, subkey_secret)) {
*error_details = "Symmetric key setup failed";
return QUIC_CRYPTO_SYMMETRIC_KEY_SETUP_FAILED;
}
string forward_secure_public_value;
if (ephemeral_key_source_.get()) {
params->forward_secure_premaster_secret =
ephemeral_key_source_->CalculateForwardSecureKey(
key_exchange, rand, clock->ApproximateNow(), public_value,
&forward_secure_public_value);
} else {
std::unique_ptr<KeyExchange> forward_secure_key_exchange(
key_exchange->NewKeyPair(rand));
forward_secure_public_value =
forward_secure_key_exchange->public_value().as_string();
/*生成共享密鑰*/
if (!forward_secure_key_exchange->CalculateSharedKey(
public_value, ¶ms->forward_secure_premaster_secret)) {
*error_details = "Invalid public value";
return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;
}
}
string forward_secure_hkdf_input;
label_len = strlen(QuicCryptoConfig::kForwardSecureLabel) + 1;
forward_secure_hkdf_input.reserve(label_len + hkdf_suffix.size());
forward_secure_hkdf_input.append(QuicCryptoConfig::kForwardSecureLabel,
label_len);
forward_secure_hkdf_input.append(hkdf_suffix);
string shlo_nonce;
shlo_nonce = NewServerNonce(rand, info.now);
out->SetStringPiece(kServerNonceTag, shlo_nonce);
/*生成密鑰*/
if (!CryptoUtils::DeriveKeys(
params->forward_secure_premaster_secret, params->aead,
info.client_nonce,
shlo_nonce.empty() ? info.server_nonce : shlo_nonce,
forward_secure_hkdf_input, Perspective::IS_SERVER,
CryptoUtils::Diversification::Never(),
¶ms->forward_secure_crypters, ¶ms->subkey_secret)) {
*error_details = "Symmetric key setup failed";
return QUIC_CRYPTO_SYMMETRIC_KEY_SETUP_FAILED;
}
out->set_tag(kSHLO);
QuicTagVector supported_version_tags;
for (size_t i = 0; i < supported_versions.size(); ++i) {
supported_version_tags.push_back(
QuicVersionToQuicTag(supported_versions[i]));
}
out->SetVector(kVER, supported_version_tags);
out->SetStringPiece(
kSourceAddressTokenTag,
NewSourceAddressToken(*requested_config.get(), info.source_address_tokens,
client_address.address(), rand, info.now, nullptr));
QuicSocketAddressCoder address_coder(client_address);
out->SetStringPiece(kCADR, address_coder.Encode());
/*server hello包中設置server的公鑰，後續client會利用這個生成對稱加密的key*/
out->SetStringPiece(kPUBS, forward_secure_public_value);
return QUIC_NO_ERROR;
}這裏用了不同的方法來生成對稱加密的 key。這裏以橢圓曲線爲例，計算對稱加密 key 的代碼如下：這是直接調用了 openssl/curve25519.h 的接口計算出來的。一旦雙方都生成了對稱密鑰，後續就可以通過對稱加密通信了！
bool Curve25519KeyExchange::CalculateSharedKey(StringPiece peer_public_value,
string* out_result) const {
if (peer_public_value.size() != crypto::curve25519::kBytes) {
return false;
}
uint8_t result[crypto::curve25519::kBytes];
if (!crypto::curve25519::ScalarMult(
private_key_,
reinterpret_cast<const uint8_t*>(peer_public_value.data()), result)) {
return false;
}
out_result->assign(reinterpret_cast<char*>(result), sizeof(result));
return true;
}
bool ScalarMult(const uint8_t* private_key,
const uint8_t* peer_public_key,
uint8_t* shared_key) {
return !!X25519(shared_key, private_key, peer_public_key);
}通信時給 packet 加密的方法：
bool AeadBaseEncrypter::EncryptPacket(QuicPathId path_id,
QuicPacketNumber packet_number,
StringPiece associated_data,
StringPiece plaintext,
char* output,
size_t* output_length,
size_t max_output_length) {
size_t ciphertext_size = GetCiphertextSize(plaintext.length());
if (max_output_length < ciphertext_size) {
return false;
}
// TODO(ianswett): Introduce a check to ensure that we don't encrypt with the
// same packet number twice.
const size_t nonce_size = nonce_prefix_size_ + sizeof(packet_number);
ALIGNAS(4) char nonce_buffer[kMaxNonceSize];
memcpy(nonce_buffer, nonce_prefix_, nonce_prefix_size_);
uint64_t path_id_packet_number =
QuicUtils::PackPathIdAndPacketNumber(path_id, packet_number);
memcpy(nonce_buffer + nonce_prefix_size_, &path_id_packet_number,
sizeof(path_id_packet_number));
/*這裏用nonce給明文加密*/
if (!Encrypt(StringPiece(nonce_buffer, nonce_size), associated_data,
plaintext, reinterpret_cast<unsigned char*>(output))) {
return false;
}
*output_length = ciphertext_size;
return true;
}最後，server hello 消息是從這裏發出去的，並且在某些情況下 server hello 已經用 server 新生成的 key 加密了，如下：
void QuicCryptoServerStream::FinishProcessingHandshakeMessage(
const ValidateClientHelloResultCallback::Result& result,
std::unique_ptr<ProofSource::Details> details) {
const CryptoHandshakeMessage& message = result.client_hello;
// Clear the callback that got us here.
DCHECK(validate_client_hello_cb_ != nullptr);
validate_client_hello_cb_ = nullptr;
if (use_stateless_rejects_if_peer_supported_) {
peer_supports_stateless_rejects_ = DoesPeerSupportStatelessRejects(message);
}
CryptoHandshakeMessage reply;
DiversificationNonce diversification_nonce;
string error_details;
QuicErrorCode error =
/*server處理client的hello消息：重點是生成對稱加密key、自己的公鑰和nonce
同時生成給client回覆的消息*/
ProcessClientHello(result, std::move(details), &reply,
&diversification_nonce, &error_details);
if (error != QUIC_NO_ERROR) {
CloseConnectionWithDetails(error, error_details);
return;
}
if (reply.tag() != kSHLO) {
if (reply.tag() == kSREJ) {
DCHECK(use_stateless_rejects_if_peer_supported_);
DCHECK(peer_supports_stateless_rejects_);
// Before sending the SREJ, cause the connection to save crypto packets
// so that they can be added to the time wait list manager and
// retransmitted.
session()->connection()->EnableSavingCryptoPackets();
}
SendHandshakeMessage(reply);//給client發server hello
if (reply.tag() == kSREJ) {
DCHECK(use_stateless_rejects_if_peer_supported_);
DCHECK(peer_supports_stateless_rejects_);
DCHECK(!handshake_confirmed());
DVLOG(1) << "Closing connection "
<< session()->connection()->connection_id()
<< " because of a stateless reject.";
session()->connection()->CloseConnection(
QUIC_CRYPTO_HANDSHAKE_STATELESS_REJECT, "stateless reject",
ConnectionCloseBehavior::SILENT_CLOSE);
}
return;
}
// If we are returning a SHLO then we accepted the handshake. Now
// process the negotiated configuration options as part of the
// session config.
//代碼到這裏已經給client發送了client hello，表示server已經準備好接受數據了
//這裏保存一些雙方協商好的通信配置
QuicConfig* config = session()->config();
OverrideQuicConfigDefaults(config);
error = config->ProcessPeerHello(message, CLIENT, &error_details);
if (error != QUIC_NO_ERROR) {
CloseConnectionWithDetails(error, error_details);
return;
}
session()->OnConfigNegotiated();
config->ToHandshakeMessage(&reply);
// Receiving a full CHLO implies the client is prepared to decrypt with
// the new server write key. We can start to encrypt with the new server
// write key. 可以開始用服務端新生成的key解密數據了
//
// NOTE: the SHLO will be encrypted with the new server write key.
/*既然在server已經生成了對稱加密的key，這裏可以用這個key加密server hello消息*/
session()->connection()->SetEncrypter(
ENCRYPTION_INITIAL,
crypto_negotiated_params_.initial_crypters.encrypter.release());
session()->connection()->SetDefaultEncryptionLevel(ENCRYPTION_INITIAL);
// Set the decrypter immediately so that we no longer accept unencrypted
// packets.
session()->connection()->SetDecrypter(
ENCRYPTION_INITIAL,
crypto_negotiated_params_.initial_crypters.decrypter.release());
if (version() > QUIC_VERSION_32) {
session()->connection()->SetDiversificationNonce(diversification_nonce);
}
SendHandshakeMessage(reply);//發送server hello
session()->connection()->SetEncrypter(
ENCRYPTION_FORWARD_SECURE,
crypto_negotiated_params_.forward_secure_crypters.encrypter.release());
session()->connection()->SetDefaultEncryptionLevel(ENCRYPTION_FORWARD_SECURE);
session()->connection()->SetAlternativeDecrypter(
ENCRYPTION_FORWARD_SECURE,
crypto_negotiated_params_.forward_secure_crypters.decrypter.release(),
false /* don't latch */);
encryption_established_ = true;
handshake_confirmed_ = true;
session()->OnCryptoHandshakeEvent(QuicSession::HANDSHAKE_CONFIRMED);
}（2）爲了防止 tcp 的隊頭阻塞，quic 在前面丟包的情況下任然繼續發包，丟的包用新的 packet number 重新發，怎麼區別這個新包是以往丟包的重發了？核心是每個包都有 stream id 和 stream offset 字段，根據這兩個字段定位包的位置，而不是 packet number。整個包結構定義的類在這裏：
struct NET_EXPORT_PRIVATE QuicStreamFrame {
QuicStreamFrame();
QuicStreamFrame(QuicStreamId stream_id,
bool fin,
QuicStreamOffset offset,
base::StringPiece data);
QuicStreamFrame(QuicStreamId stream_id,
bool fin,
QuicStreamOffset offset,
QuicPacketLength data_length,
UniqueStreamBuffer buffer);
~QuicStreamFrame();
NET_EXPORT_PRIVATE friend std::ostream& operator<<(std::ostream& os,
const QuicStreamFrame& s);
QuicStreamId stream_id;
bool fin;
QuicPacketLength data_length;
const char* data_buffer;
QuicStreamOffset offset; // Location of this data in the stream.
// nullptr when the QuicStreamFrame is received, and non-null when sent.
UniqueStreamBuffer buffer;
private:
QuicStreamFrame(QuicStreamId stream_id,
bool fin,
QuicStreamOffset offset,
const char* data_buffer,
QuicPacketLength data_length,
UniqueStreamBuffer buffer);
DISALLOW_COPY_AND_ASSIGN(QuicStreamFrame);
};收到後自然要把 payload 取出來拼接成完整的數據，stream id 和 stream offset 必不可少，拼接和處理的邏輯在這裏：裏面涉及到很多 duplicate 冗餘去重的動作，都是依據 offset 來判斷的！
QuicErrorCode QuicStreamSequencerBuffer::OnStreamData(
QuicStreamOffset starting_offset,
base::StringPiece data,
QuicTime timestamp,
size_t* const bytes_buffered,
std::string* error_details) {
*bytes_buffered = 0;
QuicStreamOffset offset = starting_offset;
size_t size = data.size();
if (size == 0) {
*error_details = "Received empty stream frame without FIN.";
return QUIC_EMPTY_STREAM_FRAME_NO_FIN;
}
// Find the first gap not ending before |offset|. This gap maybe the gap to
// fill if the arriving frame doesn't overlaps with previous ones.
std::list<Gap>::iterator current_gap = gaps_.begin();
while (current_gap != gaps_.end() && current_gap->end_offset <= offset) {
++current_gap;
}
DCHECK(current_gap != gaps_.end());
// "duplication": might duplicate with data alread filled,but also might
// overlap across different base::StringPiece objects already written.
// In both cases, don't write the data,
// and allow the caller of this method to handle the result.
if (offset < current_gap->begin_offset &&
offset + size <= current_gap->begin_offset) {
DVLOG(1) << "Duplicated data at offset: " << offset << " length: " << size;
return QUIC_NO_ERROR;
}
if (offset < current_gap->begin_offset &&
offset + size > current_gap->begin_offset) {
// Beginning of new data overlaps data before current gap.
*error_details =
string("Beginning of received data overlaps with buffered data.\n") +
"New frame range " + RangeDebugString(offset, offset + size) +
" with first 128 bytes: " +
string(data.data(), data.length() < 128 ? data.length() : 128) +
"\nCurrently received frames: " + ReceivedFramesDebugString() +
"\nCurrent gaps: " + GapsDebugString();
return QUIC_OVERLAPPING_STREAM_DATA;
}
if (offset + size > current_gap->end_offset) {
// End of new data overlaps with data after current gap.
*error_details =
string("End of received data overlaps with buffered data.\n") +
"New frame range " + RangeDebugString(offset, offset + size) +
" with first 128 bytes: " +
string(data.data(), data.length() < 128 ? data.length() : 128) +
"\nCurrently received frames: " + ReceivedFramesDebugString() +
"\nCurrent gaps: " + GapsDebugString();
return QUIC_OVERLAPPING_STREAM_DATA;
}
// Write beyond the current range this buffer is covering.
if (offset + size > total_bytes_read_ + max_buffer_capacity_bytes_) {
*error_details = "Received data beyond available range.";
return QUIC_INTERNAL_ERROR;
}
if (current_gap->begin_offset != starting_offset &&
current_gap->end_offset != starting_offset + data.length() &&
gaps_.size() >= kMaxNumGapsAllowed) {
// This frame is going to create one more gap which exceeds max number of
// gaps allowed. Stop processing.
*error_details = "Too many gaps created for this stream.";
return QUIC_TOO_MANY_FRAME_GAPS;
}
size_t total_written = 0;
size_t source_remaining = size;
const char* source = data.data();
// Write data block by block. If corresponding block has not created yet,
// create it first.
// Stop when all data are written or reaches the logical end of the buffer.
while (source_remaining > 0) {
const size_t write_block_num = GetBlockIndex(offset);
const size_t write_block_offset = GetInBlockOffset(offset);
DCHECK_GT(blocks_count_, write_block_num);
size_t block_capacity = GetBlockCapacity(write_block_num);
size_t bytes_avail = block_capacity - write_block_offset;
// If this write meets the upper boundary of the buffer,
// reduce the available free bytes.
if (offset + bytes_avail > total_bytes_read_ + max_buffer_capacity_bytes_) {
bytes_avail = total_bytes_read_ + max_buffer_capacity_bytes_ - offset;
}
if (reduce_sequencer_buffer_memory_life_time_ && blocks_ == nullptr) {
blocks_.reset(new BufferBlock*[blocks_count_]());
for (size_t i = 0; i < blocks_count_; ++i) {
blocks_[i] = nullptr;
}
}
if (blocks_[write_block_num] == nullptr) {
// TODO(danzh): Investigate if using a freelist would improve performance.
// Same as RetireBlock().
blocks_[write_block_num] = new BufferBlock();
}
const size_t bytes_to_copy = min<size_t>(bytes_avail, source_remaining);
char* dest = blocks_[write_block_num]->buffer + write_block_offset;
DVLOG(1) << "Write at offset: " << offset << " length: " << bytes_to_copy;
memcpy(dest, source, bytes_to_copy);
source += bytes_to_copy;
source_remaining -= bytes_to_copy;
offset += bytes_to_copy;
total_written += bytes_to_copy;
}
DCHECK_GT(total_written, 0u);
*bytes_buffered = total_written;
UpdateGapList(current_gap, starting_offset, total_written);
frame_arrival_time_map_.insert(
std::make_pair(starting_offset, FrameInfo(size, timestamp)));
num_bytes_buffered_ += total_written;
return QUIC_NO_ERROR;
}（3）爲了精準測量 RTT，quic 協議的數據包編號都是單調遞增的，哪怕是重發的包的編號都是增加的，這部分的控制代碼在 WritePacket 函數里面：函數開頭就判斷數據包編號。一旦發現編號比最後一次發送包的編號還小，說明出錯了，這時就關閉連接退出函數！
bool QuicConnection::WritePacket(SerializedPacket* packet) {
/*如果數據包號比最後一個發送包的號還小，說明順序錯了，直接關閉連接*/
if (packet->packet_number <
sent_packet_manager_->GetLargestSentPacket(packet->path_id)) {
QUIC_BUG << "Attempt to write packet:" << packet->packet_number << " after:"
<< sent_packet_manager_->GetLargestSentPacket(packet->path_id);
CloseConnection(QUIC_INTERNAL_ERROR, "Packet written out of order.",
ConnectionCloseBehavior::SEND_CONNECTION_CLOSE_PACKET);
return true;
}
/*沒有連接、沒有加密的包是不能發的*/
if (ShouldDiscardPacket(*packet)) {
++stats_.packets_discarded;
return true;
}
.........................
}（4）爲啥 quic 協議要基於 udp 了？應用層現成的協議很複雜，改造的難度大！傳輸層只有 tcp 和 udp 兩種協議；tcp 的缺點不再贅述，udp 的優點就是簡單，只提供最原始的發包功能，完全不管對方有沒有收到，quic 就是利用了 udp 這種最基礎的 send package 發包能力，在此之上完成了 tls（保證數據安全）、擁塞控制（保證鏈路被塞滿）、多路複用（保證數據不丟失）等應用層的功能！
本文由 Readfog 進行 AMP 轉碼，版權歸原作者所有。
來源：https://mp.weixin.qq.com/s/Qlh4_ANoFh9zFQ8EchFH3w