Repository navigation
feat(qwp): add owned_by to name the owner of the tables a sender auto-creates - #130
Open
glasstiger wants to merge 5 commits into
Open
glasstiger wants to merge 5 commits into
glasstiger wants to merge 5 commits into
Conversation
… auto-creates The owned_by connect-string key, and LineSenderBuilder.ownedBy(), name the user, group or service account that QuestDB Enterprise grants the owner permissions on every table and column the sender auto-creates, as OWNED BY does in SQL. An external (OIDC) user cannot hold those permissions itself, so it must name one of its groups to auto-create anything. - The sender sends the owner as X-QWP-Owned-By, percent-encoded UTF-8, on every upgrade, reconnects and failovers included. WebSocket only: http, tcp and udp connect strings and builders reject it. - The upgrade fails unless the server echoes the owner, so a server that predates the header fails the connection instead of creating tables without the owner. The failure is a non-421 WebSocketUpgradeException, which the connect walk treats as terminal. - A rejected upgrade now carries the plain-text body of the response as its reason, sanitized and length-capped, in WebSocketUpgradeException and in QwpAuthFailedException (getReason()), so a refused owner is told apart from a rejected credential.
The owned_by helpers were inserted between pollDurableAckRequest() and its javadoc, which then documented pollOwnedByRequest() instead. Move the javadoc back, give the new helpers their own, and attach an older javadoc that sat loose above them to setSuppressDurableAckHeader(), the method it describes.
WebSocketClient.upgrade() throws WebSocketUpgradeException with status 101 when the server completes the upgrade without echoing the owned_by the client asked for, but the class javadoc still said the exception means a non-101 status. The javadoc now names that case.
Tests now pin the parts of owned_by and of the upgrade reject reason that nothing checked before: - A reconnect and an orphan drainer's upgrade send X-QWP-Owned-By again, not only the first connect (QwpOwnedByTest, BackgroundDrainerEndToEndTest). - The reject reason stays null when the Content-Length exceeds the bytes received, is malformed or not positive, the body is not text/plain, or it is blank, and it reaches messages only sanitized and capped at 256 characters. TestWebSocketServer can now send reject headers that do not describe the body. - encodeOwnedBy keeps the ends of the unreserved ranges and encodes the characters just outside them, as the server's re-encoded echo must match it byte for byte. The owned_by echo failure now carries STATUS_NONE instead of 101: the server answered 101 and rejected nothing, so Sender.health() reports status 0 for it, as it does for the durable-ack gap and as ConnectionHealth documents. The ownedBy() javadoc no longer says the built-in admin may name any entity: Enterprise refuses an external user as the owner, the admin's choice included.
WebSocketClient's comment on the owned_by echo check said that the connect walk latches a missing echo as terminal. That holds for the initial connect and for an orphan drainer, but a sender that has connected before keeps retrying the error, as QwpOwnedByTest.testHealthReportsUnconfirmedOwnedByWithoutStatus asserts. The comment now says so, and so does the test helper that repeated it. TestWebSocketServer's suppressOwnedByEcho flag becomes isOwnedByEchoSuppressed, set with setOwnedByEchoSuppressed(), as the project names booleans with an is or has prefix.
Contributor
[PR Coverage check]😍 pass : 73 / 74 (98.65%) file detail
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to questdb/questdb#8504 and questdb/questdb-enterprise#1572; docs in questdb/documentation#652. Matching branch
ia_qwp_owned_by.Problem
QWP ingestion auto-creates the tables and columns a sender writes, and QuestDB Enterprise grants the creator the owner permissions on them. An external user, one authenticated with an OIDC token such as an Entra ID managed identity, cannot hold permissions, so it could not auto-create anything over QWP. SQL solves this with
OWNED BY; QWP had no equivalent.Changes
owned_by=<name>connect-string key (an ingress key inConfigSchema) andLineSenderBuilder.ownedBy(String).ws/wssonly:http,tcpandudpconnect strings and builders reject it. TheQwpQueryClientaccepts it as a no-op, like the other ingress keys.X-QWP-Owned-Byon every upgrade, reconnects, failovers and background drainers included. The value is percent-encoded UTF-8 (RFC 3986 unreserved characters as they are, upper-case hex digits), so names outside ASCII survive.WebSocketUpgradeException, which the connect walk already treats as terminal. It carries no HTTP status (STATUS_NONE), as the server answered 101, so connection health reports none, as for a missing durable-ack echo.CredentialRedaction, appended to theWebSocketUpgradeExceptionmessage and exposed asQwpAuthFailedException.getReason(). A refused owner (403) readsauth-rejected: ... [reason=owned_by refused: bob is not part of group team]instead of looking like a rejected credential.QwpWebSocketSender.connectWithCredentialSupplier(...)overload withownedBy; the previous master delegates to it withnull.Tests
QwpOwnedByTest: the header and its canonical encoding, including the ends of the unreserved ranges, the echo accepted, missing and mismatched, the header sent again on a reconnect, the reject reason with and without a body, its absence when the body is unusable (a Content-Length beyond the bytes received, malformed or not positive, another content type, a blank body), its sanitizing and length cap, both sender build paths (connect string and builder), a 403 refusal surfacing asQwpAuthFailedExceptionwith its reason, and connection health on a missing echo.BackgroundDrainerEndToEndTest: an orphan drainer's upgrade names the owner too.LineSenderBuilderWebSocketTest,WsSenderConfigHonoredTestandQwpQueryClientFromConfigTest: parsing, validation and the key-registry drift guard.