Skip to content

feat(qwp): add owned_by to name the owner of the tables a sender auto-creates - #130

Open
glasstiger wants to merge 5 commits into
mainfrom
ia_qwp_owned_by
Open

glasstiger wants to merge 5 commits into
mainfrom
ia_qwp_owned_by

Conversation

@glasstiger

@glasstiger glasstiger commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Companion to questdb/questdb#8504 and questdb/questdb-enterprise#1572; docs in questdb/documentation#652. Matching branch ia_qwp_owned_by.

Problem

QWP ingestion auto-creates the tables and columns a sender writes, and QuestDB Enterprise grants the creator the owner permissions on them. An external user, one authenticated with an OIDC token such as an Entra ID managed identity, cannot hold permissions, so it could not auto-create anything over QWP. SQL solves this with OWNED BY; QWP had no equivalent.

Changes

  • owned_by=<name> connect-string key (an ingress key in ConfigSchema) and LineSenderBuilder.ownedBy(String). ws/wss only: http, tcp and udp connect strings and builders reject it. The QwpQueryClient accepts it as a no-op, like the other ingress keys.
  • The sender sends it as X-QWP-Owned-By on every upgrade, reconnects, failovers and background drainers included. The value is percent-encoded UTF-8 (RFC 3986 unreserved characters as they are, upper-case hex digits), so names outside ASCII survive.
  • The upgrade fails unless the server echoes the owner: a server that predates the header ignores it, and the tables it creates would not get the owner. The failure is a non-421 WebSocketUpgradeException, which the connect walk already treats as terminal. It carries no HTTP status (STATUS_NONE), as the server answered 101, so connection health reports none, as for a missing durable-ack echo.
  • A rejected upgrade now carries the plain-text body of the response as its reason, when it arrived with the headers: length-capped, sanitized with CredentialRedaction, appended to the WebSocketUpgradeException message and exposed as QwpAuthFailedException.getReason(). A refused owner (403) reads auth-rejected: ... [reason=owned_by refused: bob is not part of group team] instead of looking like a rejected credential.
  • New master QwpWebSocketSender.connectWithCredentialSupplier(...) overload with ownedBy; the previous master delegates to it with null.

Tests

  • QwpOwnedByTest: the header and its canonical encoding, including the ends of the unreserved ranges, the echo accepted, missing and mismatched, the header sent again on a reconnect, the reject reason with and without a body, its absence when the body is unusable (a Content-Length beyond the bytes received, malformed or not positive, another content type, a blank body), its sanitizing and length cap, both sender build paths (connect string and builder), a 403 refusal surfacing as QwpAuthFailedException with its reason, and connection health on a missing echo.
  • BackgroundDrainerEndToEndTest: an orphan drainer's upgrade names the owner too.
  • LineSenderBuilderWebSocketTest, WsSenderConfigHonoredTest and QwpQueryClientFromConfigTest: parsing, validation and the key-registry drift guard.
  • The server side is covered end to end in the OSS and Enterprise PRs.

… auto-creates

The owned_by connect-string key, and LineSenderBuilder.ownedBy(), name the user,
group or service account that QuestDB Enterprise grants the owner permissions on
every table and column the sender auto-creates, as OWNED BY does in SQL. An
external (OIDC) user cannot hold those permissions itself, so it must name one of
its groups to auto-create anything.

- The sender sends the owner as X-QWP-Owned-By, percent-encoded UTF-8, on every
  upgrade, reconnects and failovers included. WebSocket only: http, tcp and udp
  connect strings and builders reject it.
- The upgrade fails unless the server echoes the owner, so a server that
  predates the header fails the connection instead of creating tables without
  the owner. The failure is a non-421 WebSocketUpgradeException, which the
  connect walk treats as terminal.
- A rejected upgrade now carries the plain-text body of the response as its
  reason, sanitized and length-capped, in WebSocketUpgradeException and in
  QwpAuthFailedException (getReason()), so a refused owner is told apart from a
  rejected credential.
The owned_by helpers were inserted between pollDurableAckRequest() and
its javadoc, which then documented pollOwnedByRequest() instead. Move
the javadoc back, give the new helpers their own, and attach an older
javadoc that sat loose above them to setSuppressDurableAckHeader(),
the method it describes.
WebSocketClient.upgrade() throws WebSocketUpgradeException with status
101 when the server completes the upgrade without echoing the owned_by
the client asked for, but the class javadoc still said the exception
means a non-101 status. The javadoc now names that case.
Tests now pin the parts of owned_by and of the upgrade reject reason
that nothing checked before:

- A reconnect and an orphan drainer's upgrade send X-QWP-Owned-By
  again, not only the first connect (QwpOwnedByTest,
  BackgroundDrainerEndToEndTest).
- The reject reason stays null when the Content-Length exceeds the
  bytes received, is malformed or not positive, the body is not
  text/plain, or it is blank, and it reaches messages only sanitized
  and capped at 256 characters. TestWebSocketServer can now send
  reject headers that do not describe the body.
- encodeOwnedBy keeps the ends of the unreserved ranges and encodes
  the characters just outside them, as the server's re-encoded echo
  must match it byte for byte.

The owned_by echo failure now carries STATUS_NONE instead of 101: the
server answered 101 and rejected nothing, so Sender.health() reports
status 0 for it, as it does for the durable-ack gap and as
ConnectionHealth documents.

The ownedBy() javadoc no longer says the built-in admin may name any
entity: Enterprise refuses an external user as the owner, the admin's
choice included.
WebSocketClient's comment on the owned_by echo check said that the
connect walk latches a missing echo as terminal. That holds for the
initial connect and for an orphan drainer, but a sender that has
connected before keeps retrying the error, as
QwpOwnedByTest.testHealthReportsUnconfirmedOwnedByWithoutStatus
asserts. The comment now says so, and so does the test helper that
repeated it.

TestWebSocketServer's suppressOwnedByEcho flag becomes
isOwnedByEchoSuppressed, set with setOwnedByEchoSuppressed(), as the
project names booleans with an is or has prefix.
@mtopolnik

Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 73 / 74 (98.65%)

file detail

path covered line new line coverage
🔵 io/questdb/client/Sender.java 16 17 94.12%
🔵 io/questdb/client/cutlass/http/client/WebSocketClient.java 46 46 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpAuthFailedException.java 6 6 100.00%
🔵 io/questdb/client/impl/ConfigSchema.java 1 1 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpWebSocketSender.java 3 3 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpUpgradeFailures.java 1 1 100.00%

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants