Add socket reserve/use connection affinity + fd_version primitives#3348
Closed
rajvarun77 wants to merge 1 commit into
Closed
Add socket reserve/use connection affinity + fd_version primitives#3348rajvarun77 wants to merge 1 commit into
rajvarun77 wants to merge 1 commit into
Conversation
Introduces generic Controller/Socket plumbing that lets a protocol pin a pooled/short connection across multiple RPCs (connection affinity), needed by request/response protocols with per-connection server state such as MySQL interactive transactions and prepared statements. No protocol uses it yet; this is the shared substrate for the MySQL client PR (apache#3330), split out for independent review. Controller: - BindSockAction {NONE,RESERVE,USE} encoded in two Controller::_flags bits (FLAGS_BIND_SOCK_RESERVE/USE) via set_bind_sock_action()/bind_sock_action(); no dedicated member on the hot Controller. - _bind_sock holds a socket reserved by a previous RPC; EndRPC propagates the action to the current Call, and Call::OnComplete reserves the socket (on success) instead of returning it to the pool / failing it. IssueRPC reuses the reserved socket when the action is USE. - _session_data: opaque per-RPC slot a protocol codec may use to carry typed state between serialize/pack/parse (not owned by Controller). Socket: - _fd_version (relaxed atomic, bumped on every ResetFileDescriptor) plus a fd_version() accessor, an ABA guard so an affinity holder can detect that a reserved fd was reconnected underneath it. Atomic because it is written on the reconnect path (no socket lock held) and read from other threads. - Socket::Write accepts an empty buffer when WriteOptions::auth_flags is set, for server-greets-first handshakes that send their first bytes from the connection-phase handler rather than from `data`. Also fixes an uninitialized read this machinery would otherwise introduce: the Call(Call*) copy constructor used for backup requests and retries did not initialize bind_sock_action, so OnComplete could read indeterminate bits and (when they matched RESERVE/USE) divert the backup call's socket away from the pool-return path, hanging the RPC. Initialize to BIND_SOCK_NONE, matching Call::Reset(); a backup/retry never inherits affinity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
This is the shared-core substrate extracted from #3330 (MySQL client protocol), which is now stacked on top of this PR. Reviewing this first keeps the generic |
Contributor
Author
|
Reverting the split — folding the backup/retry copy-constructor fix back into #3330 directly; a separate core PR isn't warranted for a one-line initialization fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generic
Controller/Socketplumbing that lets a protocol pin a pooled/short connection across multiple RPCs (connection affinity). This is the shared substrate extracted from the MySQL client PR #3330 so it can be reviewed independently; #3330 is now stacked on top of this branch and contains only the MySQL-specific code.Controller
BindSockAction {NONE,RESERVE,USE}encoded in twoController::_flagsbits (FLAGS_BIND_SOCK_RESERVE/USE) viaset_bind_sock_action()/bind_sock_action()— no dedicated member added to the hotController._bind_sockholds a socket reserved by a previous RPC.EndRPCpropagates the action to the currentCall;Call::OnCompletereserves the socket (on success) instead of returning it to the pool / failing it;IssueRPCreuses the reserved socket when the action isUSE._session_data: opaque per-RPC slot a protocol codec may use to carry typed state between serialize/pack/parse (not owned byController).Socket
_fd_version(relaxed atomic, bumped on everyResetFileDescriptor) +fd_version()accessor — an ABA guard so an affinity holder can detect that a reserved fd was reconnected underneath it. Atomic because it is written on the reconnect path (no socket lock held) and read from other threads.Socket::Writeaccepts an empty buffer whenWriteOptions::auth_flagsis set, for server-greets-first handshakes that send their first bytes from the connection-phase handler rather than fromdata.Backup/retry fix
Also fixes an uninitialized read this machinery would otherwise introduce: the
Call(Call*)copy constructor used for backup requests and retries did not initializebind_sock_action, soOnCompletecould read indeterminate bits and (when they matchedRESERVE/USE) divert the backup call's socket away from the pool-return path, hanging the RPC (manifested asChannelTest.backup_requesttiming out). Initialized toBIND_SOCK_NONE, matchingCall::Reset(); a backup/retry never inherits affinity.No protocol uses these primitives in this PR; the first consumer is MySQL (#3330).
🤖 Generated with Claude Code