Skip to content

feat(qwp): add paging, row counts, timings and error positions to the query client - #129

Open
glasstiger wants to merge 9 commits into
mainfrom
ia_query_info
Open

glasstiger wants to merge 9 commits into
mainfrom
ia_query_info

Conversation

@glasstiger

@glasstiger glasstiger commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

PR #129 adds server-side paging and /exec-style query metadata to the Java QWP query client.

New functionality

A new QwpQueryOptions class lets callers request:

  • Paging: skipRows and maxRows select a window of results. maxRows(0) returns only the schema.
  • Full-result row counts: countRows(true) counts rows beyond the requested page, separately from the number actually streamed. This can require the server to read the remaining results.
  • Execution statistics: compilation, total execution, counting and client-wait times in nanoseconds, plus whether the filter used JIT compilation.
  • SQL error positions: errorPosition(true) provides the offending character’s position separately from the error message.

The options also expose the existing SYMBOL-dictionary reset functionality.

API integration

  • Adds options-aware QwpQueryClient.execute(...) overloads and Query.options(...) on the higher-level facade.
  • Adds QwpQueryDetails to new onEnd(...) and onExecDone(...) callback overloads, and a position argument to onError(...).
  • Existing handlers continue working through default-method delegation. Client-side failures also reach the position-aware error callback.
  • QueryException.getPosition() exposes the position when Completion.await() fails; unavailable positions use -1.
  • The facade snapshots options at submit(), so preparing the next page cannot alter an in-flight query. Failover retries retain the original options.
  • Details objects are pooled and valid only during the callback; callers must copy values they want to retain.

Compatibility and implementation

  • Queries without options remain byte-identical on the wire.
  • The client checks the server’s CAP_QUERY_DETAILS capability, including after failover. Unsupported metadata requests run without metadata; unsupported paging fails before sending, rather than unexpectedly returning every row.
  • The new protocol bits avoid those already used for per-query timeouts by released C/Python clients; this PR does not add Java timeout support.
  • Response decoding gains a shared bounds-checked varint reader, with stricter rejection of truncated fields and overflow.
  • Successful terminal events now use the event pool, avoiding fresh event/details allocations for each completion.

The PR also adds a README example and extensive tests covering encoding, decoding, compatibility, callback delegation, option snapshots, failover/shutdown failures and pooled-state reuse.

Server counterpart: questdb/questdb#8356.

QwpQueryClient and the Query facade can now use the query-details
flags of QUERY_REQUEST, which the server advertises with the
CAP_QUERY_DETAILS bit of SERVER_INFO (questdb PR #8356):

- QwpQueryOptions sets a row window (skipRows, maxRows) and asks for
  the row count of the whole result set (countRows), execution
  statistics (stats) and the error position (errorPosition). It also
  carries resetSymbolDict. QwpQueryClient.execute() and
  Query.options() take it; the client reads it once, when the query
  starts, so a failover replay sends the same options.
- The handler receives the answers through new default overloads:
  onEnd and onExecDone with a QwpQueryDetails (the row count, the
  compile, execute, count and client-wait times, and the JIT flag),
  and onError with the position. Each delegates to the next simpler
  overload, so existing handlers keep working. The client now calls
  the detailed overloads for every terminal event, client-side
  failures included, so one override sees them all.
- QueryException.getPosition() carries the position to callers of
  the facade's Completion.await().

A query without options is byte-identical on the wire. Against a
server without CAP_QUERY_DETAILS, the client leaves out the count,
the statistics and the position, and fails a query that sets a row
window before sending it, since that server would stream every row.
After a failover reconnect the client checks the new server again.

The I/O thread decodes the trailers the request asked for: row_count
and the statistics block on RESULT_END, the statistics block on
EXEC_DONE, and the position on QUERY_ERROR. A RESULT_END or EXEC_DONE
without a requested trailer is a terminal transport error, like any
other truncated terminator. A QUERY_ERROR without the position is the
baseline frame the server sends for a request whose flags it could
not read; it reports no position. RESULT_END and EXEC_DONE now share
one bounds-checked varint reader, which also rejects data bits above
bit 63 on a tenth byte, as the result batch decoder already does.
@glasstiger glasstiger added enhancement New feature or request tandem QWP labels Nov 12, 2026
The query-details change routes every failure that execute() detects
itself to the position-aware onError overload, but the tests written
for it reached none of those paths. QwpQueryClientFailurePathsTest now
drives each one with a handler that fails on any other overload:

- a failover reconnect that fails, and the reconnect of the next query
  that fails after it;
- a stop request between two attempts, from onFailoverReset;
- a stop request while the failover reconnect waits for SERVER_INFO;
- a stop request right after a transport failure;
- a close() between two attempts, from onFailoverReset.

A stop request can reach the point right after a transport failure
only through a race with another thread, so QwpQueryClient gains
setAfterTransportFailureHookForTest(), a barrier that runs there, like
setBeforeCloseHookForTest(). TestWebSocketServer gains a hook that runs
before SERVER_INFO, and its rejected egress upgrades now drop the
connection, as its documentation says, instead of leaving the client to
wait out its auth timeout.

The default branch of the event switch in executeOnce() is unreachable,
as the I/O thread emits only the kinds it handles; an assertion now
says so. A comment explains the null guard on serverInfo, which only a
connect() racing the query from another thread can trip.
glasstiger added a commit to questdb/questdb that referenced this pull request Nov 12, 2026
QwpEgressQueryDetailsClientTest sends the new query_flags through the
real java-questdb-client, with QwpQueryOptions on both
QwpQueryClient.execute() and the facade's Query.options(). It checks
the error positions that onError() and QueryException receive, the row
window, the full row count, the timings and the JIT flag against /exec
on the same server and against SQL LIMIT, and asserts that the client's
wire constants match the server's.

The released client 1.3.12 cannot send these flags, so the submodule
moves to the head of questdb/java-questdb-client#129 and the poms pin
questdb.client.version to 1.3.13-SNAPSHOT. CI then builds the client
from the submodule with the local-client profile.

The Javadoc of QwpEgressQueryDetailsWireTest no longer claims that the
pinned client cannot send the flags.
The javadoc of QwpColumnBatchHandler.onError(long, byte, String, int)
and of QwpEgressMsgKind.NO_ERROR_POSITION said that a cancellation
reports no error position. Only a CANCEL this client sends does: the
server reports position 0 for a query timeout and for a cancellation
on the server, such as CANCEL QUERY, as the /exec HTTP endpoint does.
The javadoc now says so.
The query details took SERVER_INFO capability 0x08 and query flag 0x02,
which per-query timeouts already use in released clients:
c-questdb-client 7.2.0 and later, and the Python client 6.0.0 and later,
read capability 0x08 as "the server honours a per-query timeout" and
then send flag 0x02 followed by timeout_ms. A server with the query
details would have read that flag as an error-position request, ignored
the timeout and answered with a QUERY_ERROR those clients cannot decode.

CAP_QUERY_DETAILS moves to 0x10, and the flags to 0x04
(ERROR_POSITION), 0x08 (STATS), 0x10 (COUNT_ROWS), 0x20 (SKIP_ROWS) and
0x40 (MAX_ROWS), as on the server. CAP_QUERY_TIMEOUT (0x08) and
QUERY_FLAG_TIMEOUT (0x02) now name the reserved bits, with the values
the timeout change gives them, and the docs say a flag's field goes
after query_flags in ascending order of the flag bits, so timeout_ms
comes ahead of skip_rows and max_rows.

New tests pin the allocation: every capability and flag is a bit of its
own and none takes the timeout bits. A server that advertises per-query
timeouts without the query details gets none of the details, and a
row window fails before it is sent.
QwpQueryClient reported the failures execute() detects itself through
eleven separate handler.onError() calls. Five of them -- an exhausted
failover (two sites), an interrupted backoff, and a rejected or
unavailable credential during a failover reconnect -- had no test that
would notice a call to the 3-arg overload. A regression there would
hide the error from a handler that overrides only
onError(long, byte, String, int), as QwpColumnBatchHandler tells users
to do. deliverClientError() now makes that call for every
client-detected failure, deliverFinal() included, so any test that
reaches one of them pins all of them. QwpQueryClientFailurePathsTest
adds the exhausted failover, the rejected credential and the
unavailable credential; QwpEgressMockServer gains
setAuthorizationValidator() for the rejected credential.

QuestDBQueryOptionsTest now checks that options() on a closed handle
throws and leaves the next borrower's query alone. No test covered the
stale-lease guard of the new Query.options() before.

Test helper cleanup:
- RecordingQueryHandler drops the unread batchCount and hosts the
  assertNoDetails(QwpQueryDetails) that QueryEventTest and
  QwpEgressTerminatorDecodeTest each carried a copy of.
- QwpEgressFrames.QueryRequest drops the unread initialCredit.
- testEachDetailOptionSendsItsFlag becomes
  testEachBooleanOptionSendsItsFlag: resetSymbolDict is not a
  query-details option.
- The QueryException test moves from QwpQueryOptionsTest to its own
  QueryExceptionTest.
- QwpEgressMockServer drops a connection on its read thread. The
  comment that sent the close to another thread claimed close() joins
  the read thread, which it skips when called on that thread.
Query.options() wrote into the handle's options, which the worker
thread read only when it started the query, after submit() had
returned. A caller that set up the next page before await() changed
the query already in flight: it streamed the next window, or every
row after options(null). submit() now copies the options into
submittedOptions, which execute() reads, so a change made while a
query is in flight applies to the next submit() only. The Query
javadoc says so, and tells callers not to change the SQL text, binds
or handler before await() returns, as the running query reads them.
The comment in QueryImpl.runOn() no longer claims that nothing can
call sql() while a query runs.

QueryClientPool gains setBusyWorkerHookForTesting(), which installs
the existing busyWorkerTestHook on its workers. The new
QuestDBQueryOptionsTest test uses it to park the worker between two
queries, so the caller always changes the options before the worker
reads them.

RecordingQueryHandler failed a call to a simpler overload by throwing,
which the facade's worker thread swallows once WrappingHandler has
signalled success. A WrappingHandler that also called a legacy
terminal overload passed QuestDBQueryOptionsTest. The handler now
records such a call too, and the facade tests assert the record on the
test thread after await(); assertOnlyError() and assertNoDetails()
check it as well.
execute() reports an interrupt during the failover backoff through
deliverClientError(), but no test reached that call: with it reverted
to the 3-arg onError(), the whole suite still passed, so a regression
would hide the error from a handler that overrides only
onError(long, byte, String, int).

QwpQueryClientFailurePathsTest now drives that exit. Its hook closes
the client and then interrupts the caller, so the pending interrupt
ends the backoff's sleep rather than the join() of the failed
connection's I/O thread in the failover cleanup. A seeded random makes
that sleep 13.9 s, which the interrupt cuts short. The test fails with
the call site reverted.

QuestDBQueryOptionsTest.testErrorPositionReachesQueryException said
its last block checks a failure after one with a position, but the
failure before that block carried none. The comment now sits on the
block it describes. The last block asks for the position and checks
that a failure the client detects itself reports none, with
STATUS_INTERNAL_ERROR.
execute() reports a failover deadline that runs out while it tears
down the failed connection through deliverClientError(), but no test
reached that call: with it reverted to the 3-arg onError(), the whole
suite still passed, so a regression would hide the error from a
handler that overrides only onError(long, byte, String, int).

The window lies between the attempt check, which lets the query fail
over, and the backoff, which finds the deadline gone. QwpQueryClient
gains setAfterFailoverCleanupHookForTest(), a barrier that runs there,
like setAfterTransportFailureHookForTest().

QwpQueryClientFailurePathsTest now drives that exit: its hook waits
out a 2 s failover_max_duration_ms, and the test checks that the query
ends with no position and that the next query reconnects, as this exit
leaves the client disconnected rather than latched. The test fails
with the call site reverted.
@glasstiger

Copy link
Copy Markdown
Contributor Author

Reviewing PR #129 at level 3 (full review, with an independent check of every candidate issue). Base is 0f6c100c (main), head is d312f4bd. No submodule pointer moved and no binaries were committed.

Critical

None.

Moderate

None.

Minor

New test methods break the alphabetical member order

  • Problem: New test methods are inserted out of alphabetical order.
  • Net impact: No user impact. An IDE rearrange will later reshuffle both files.
  • Evidence: I compared the method order at head d312f4bd with base 0f6c100c. The repo's committed .idea/codeStyles/Project.xml sorts members BY_NAME.

Where:

  • QwpColumnBatchHandlerTest.java:50: testDetailedOverloadsDelegateToSimplerOnes comes before testDefaultMethodsAreNoOpAndDoNotThrow (line 105). "testDef…" sorts before "testDet…", so it should come second. At base the file had only the second method.
  • QueryEventTest.java:59, 68, 83: testAsEndAndAsExecDoneDropStaleDetails, testAsErrorWithPosition and testWithRowCountAndStats sit between testFreshEventDefaults and testAsBatchSetsBufferAndKind. testWithRowCountAndStats belongs last.
  • QueryEventTest.java:200: testResetClearsDetails sits between …FromError and …FromBatch. It belongs before every testResetClearsEverything* method.
  • QueryEventTest was already partly unsorted at base, but each of these four placements is new.

Fix: run IntelliJ's Rearrange Code on both files.

Coverage map

The test gate passes. No coverage gaps were admitted.

  • Mutation check: I changed 11 of the riskiest production lines, one at a time, and each change made at least one test fail. The lines were:
    • the fallback for a QUERY_ERROR without a position
    • the bit-63 varint check
    • the row-window refusal
    • submittedOptions vs options in QueryImpl.runOn
    • the order of skip_rows and max_rows on the wire
    • whether a decode failure counts as a transport error or a query error
    • deliverClientError calling the 4-argument onError
    • currentQueryFlags
    • passing the error position through the facade
    • the capability mask for the detail flags
    • decoding stats on EXEC_DONE
  • Test runs at head (JDK 17): the 87 new and changed tests pass, and so do all 943 tests in the Qwp*, Query* and QuestDB* classes.
  • Server tandem: feat(qwp): add error positions, timings, row counts and paging to QWP queries questdb#8356 is on the matching branch ia_query_info, and the two PRs link each other.
    • It pins this client at 17952bfe, one commit behind head. That last commit only adds a test hook and its test.
    • Its QwpEgressQueryDetailsClientTest runs the pinned client against a real server. It checks that the constants match, error positions match /exec, facade options work, row windows and counts match SQL LIMIT, and stats are reported.
  • Enterprise tandem: not needed. The changes to the failover loop only reroute error delivery and re-check server capabilities on each attempt. The local two-server failover tests cover both. Reconnect, replay and role handling are unchanged.

Summary

  • Verdict: approve.
  • Gates: no Critical findings, and the test gate passes.
  • Findings: 0 Critical, 0 Moderate, 1 Minor, all inside the diff. I found nothing broken at callers outside the diff. That check covered:
    • every handler implementation in this repo, its examples, questdb and questdb-ent
    • how the new execute overloads resolve alongside the old ones
    • the callers of submitQuery and the decode* methods
    • the client's wire layout against the tandem server's request decoder and frame writers
  • Java 8: CI's JDK 8 jobs were still pending. All main and test sources compile with javac --release 8 (with sun.misc stubbed).
  • Deliberate, documented tradeoffs:
    • Against a server without CAP_QUERY_DETAILS, a query with skipRows or maxRows fails before it is sent. Row count, stats and error position are silently absent. This also applies after a failover to an older node during a rolling upgrade.
    • Query gains an abstract options() method, as it did serverInfo() before.
  • Conventions: the PR title, body, labels and commit messages follow them.

Each QueryEvent owns a QwpQueryDetails, and the I/O thread built every
RESULT_END and EXEC_DONE event with new QueryEvent(). Every query, with
or without options, so allocated a details object it did not need: the
I/O thread allocated 144 bytes per query, up from 80 before the query
details.

The I/O thread now takes those events from its event pool, as it does
for batches; the user thread already hands every event back to the
pool after the handler. decodeResultEnd() and decodeExecDone() decode
into the event they get. An event borrowed for a frame that fails to
decode stays out of the pool, as only the user thread returns events to
it. A query that ends with an EXEC_DONE now allocates 32 bytes on the
I/O thread.

testDetailsAreReusedAcrossQueries checks that once the pool has gone
round, no query brings a QwpQueryDetails the handler has not seen, and
that an instance which carried details reports none for a query that
asked for none.

QueryEventTest and QwpColumnBatchHandlerTest now sort their tests by
name, and the class javadoc of QwpColumnBatchHandlerTest also covers the
delegation of the detailed overloads it tests.
@mtopolnik

Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 284 / 286 (99.30%)

file detail

path covered line new line coverage
🔵 io/questdb/client/cutlass/qwp/client/QwpQueryClient.java 61 63 96.83%
🔵 io/questdb/client/QueryException.java 6 6 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpQueryDetails.java 45 45 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpQueryOptions.java 55 55 100.00%
🔵 io/questdb/client/impl/QueryClientPool.java 5 5 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QueryEvent.java 14 14 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpColumnBatchHandler.java 6 6 100.00%
🔵 io/questdb/client/cutlass/qwp/client/QwpEgressIoThread.java 65 65 100.00%
🔵 io/questdb/client/impl/QueryLease.java 2 2 100.00%
🔵 io/questdb/client/impl/QueryImpl.java 25 25 100.00%

@glasstiger

Copy link
Copy Markdown
Contributor Author

Reviewing PR #129 at level 3, from base 0f6c100c (main) to head edb1c472. The head now includes edb1c472 ("Reuse pooled events to end a query"), which the earlier review on the PR (at d312f4bd) did not cover.

Critical

None.

Moderate

None.

Minor

None. The test-method ordering flagged in the earlier review is fixed at head: every changed test class is now sorted.

Coverage map

The test gate passes, with no admitted coverage gaps.

  • Local runs at edb1c472 (JDK 17): the 12 related test classes pass, 135 tests in all. These are the new and changed ones, plus QwpResultBatchDecoderHardeningTest, QueryWorkerTest and QuestDBBuilderTest.
  • Pooled end-of-query events (the new commit):
    • testDetailsAreReusedAcrossQueries checks that the details objects are reused. It would fail if the client went back to new QueryEvent().
    • testDetailsDoNotLeakToTheNextQuery checks that one query's details never reach the next.
  • Malformed end-of-query frames: these must still count as transport errors. Three tests in QwpResultBatchDecoderHardeningTest cover this, and they fail if either catch block in decodeAndEmit* reports a per-query error instead. I confirmed that in a scratch worktree.
  • CI at edb1c472:
    • JDK 8 build and test (x64 and ARM64) pass.
    • The JDK 25/26 compile and javadoc jobs pass.
    • Azure Linux, macOS and Windows pass.
    • The coverage check shows 284 of 286 changed lines covered.
  • OSS server tandem, feat(qwp): add error positions, timings, row counts and paging to QWP queries questdb#8356:
    • It is on the matching branch ia_query_info, and the two PRs link each other.
    • Its QwpEgressQueryDetailsClientTest runs the client against a real server.
    • It now pins this client at head edb1c472. Its CI on that pin has passed every job except the Griffin tests on linux-arm64, which are still running.
    • The server writes trailers that match the client: row_count then the stats on RESULT_END, only the stats on EXEC_DONE, and an i32 position on QUERY_ERROR. Every path that sends RESULT_END goes through writeResultEndBody, so a requested trailer is never missing. The tandem's newest commits only change server-side read-ahead for backward scans and leave these writers alone.
  • Enterprise tandem: not needed.
    • The failover changes only reroute how errors reach the handler, and re-check the server's capabilities on each attempt.
    • Enterprise uses the OSS egress processor unchanged.
    • The local two-server failover tests cover both changes.

Summary

  • Verdict: approve. No Critical findings, and the test gate passes.
  • Findings: 0 Critical, 0 Moderate, 0 Minor, so nothing in the diff or at callers outside it.
  • Callers checked outside the diff:
    • every QwpColumnBatchHandler in this repo, its examples, questdb (tests and benchmarks), and questdb-ent (including the e2e sidecar);
    • how the new execute overloads resolve alongside the old ones;
    • who implements Query (only QueryLease);
    • who calls submitQuery and the decode* methods (only the client itself and its tests).
  • Submodule pointers: none moved, and no binaries are committed.
  • Deliberate, documented tradeoffs:
    • Older servers: against a server without CAP_QUERY_DETAILS, a query with skipRows/maxRows fails before it is sent. The other options are silently left out. This also happens after a failover to an older node during a rolling upgrade.
    • Details are reused: after edb1c472, the QwpQueryDetails passed to a handler is reused. A handler that keeps it past the callback will see other values, as the javadoc warns.
    • API change: Query gains an abstract options() method. Nothing in this repo, questdb or questdb-ent implements Query apart from QueryLease.
    • Missing trailer: if a RESULT_END or EXEC_DONE lacks a requested trailer, the client treats it as a transport error, the same as other truncated frames. No supported server sends such a frame.
  • Conventions: the title, body, labels, commit messages, member ordering and Java 8 compatibility all follow them.

@glasstiger

Copy link
Copy Markdown
Contributor Author

Reviewing PR #129 at level 3. The base is 0f6c100c (main) and the head is edb1c472.

Critical

None.

Moderate

None.

Minor

None.

Coverage map

The test gate passes, with no coverage gaps found.

  • Local run: at edb1c472 (JDK 17, macOS arm64), all 88 tests pass across the 9 new and changed test classes.
  • CI at edb1c472: everything passes. That includes JDK 8 build, test and javadoc on x64 and ARM64, the JDK 25 and 26 compile and javadoc jobs, the multi-release JAR smoke jobs, and Azure on Linux, macOS and Windows. The coverage check covers 284 of 286 changed lines.
  • Server tandem, feat(qwp): add error positions, timings, row counts and paging to QWP queries questdb#8356:
    • It is on the matching branch ia_query_info, the two PRs link each other, and it pins this client at edb1c472.
    • Its QwpEgressQueryDetailsClientTest drives that client against a real server. It checks row windows and counts against SQL LIMIT, stats on both end-of-query frames, error positions against /exec, and QueryException.getPosition() through the pooled QuestDB client (borrowQuery).
    • Every CI job on it passes.
  • Enterprise tandem: not needed. The changes to the failover loop only route client-side errors through the 4-arg onError and re-check the server's capabilities on each attempt. Local tests run the real failover loop against two mock servers, one with the capability and one without. The diff does not touch the store-and-forward drainer, pool startup, lazy_connect or role gating.

Summary

  • Verdict: approve. There are no Critical findings and the test gate passes.
  • Findings: 0 Critical, 0 Moderate, 0 Minor. Nothing in the diff or at its callers is broken.
  • Callers checked outside the diff:
    • Every QwpColumnBatchHandler in this repo, its examples, the questdb tandem tests and benchmarks, and the questdb-ent tests and e2e sidecar. Handlers that override only the older overloads still get exactly one terminal callback through the default methods.
    • Only QueryLease implements Query.
    • Nothing outside this repo calls submitQuery or the decoders.
    • Existing execute(...) calls resolve to the same overloads as before.
  • Wire agreement with the tandem server (dc0bff0a):
    • The server reads skip_rows and then max_rows after query_flags, as the client writes them.
    • RESULT_END carries row_count and then the stats when requested, and EXEC_DONE carries only the stats.
    • QUERY_ERROR ends with an i32 position. The server sends a frame without it when it rejects a request before reading its flags, and the client handles that frame.
    • The server sets CAP_QUERY_DETAILS for every SERVER_INFO provider, so Enterprise nodes advertise it too.
  • Bit allocation: nothing collides with released clients. c-questdb-client 7.3.0 uses only capabilities 0x01–0x08 and query flags 0x01–0x02.
  • Submodules and binaries: no submodule pointer moved and no binaries are committed.
  • Tradeoffs worth knowing about (documented, not defects):
    • Older servers: against a server without CAP_QUERY_DETAILS, a query with skipRows or maxRows fails before it is sent. Count, stats and position are silently left out. This also happens after a failover to an older node during a rolling upgrade, after the handler has seen onFailoverReset.
    • Reused details: the client reuses the QwpQueryDetails passed to a handler. It is valid only during the callback.
    • Error message format: with errorPosition(true), error messages arrive without the [n] prefix.
    • Missing trailer: a RESULT_END or EXEC_DONE that lacks a requested trailer is treated as a terminal transport error. The tandem server always writes the trailers a request asks for.
    • Stricter varint check: a tenth varint byte with bits above bit 63 is now rejected. A correct encoder never produces one.
    • API change: Query gains an abstract options(), as it did serverInfo() before.
  • Conventions: the title, body, labels, commit messages, member ordering and the Java 8 floor all follow them.

@glasstiger

Copy link
Copy Markdown
Contributor Author

Reviewing PR #129 at level 3 (full review): base 0f6c100c (main), head edb1c472.

Verdict: approve. I found no Critical, Moderate or Minor issues, and the test gate passes.

Critical

None.

Moderate

None.

Minor

None.

Coverage map

The test gate passes with no coverage gaps worth reporting.

  • Local runs at edb1c472 (JDK 17, macOS arm64): 116 tests pass across 11 classes. These are the new and changed test classes plus QwpResultBatchDecoderHardeningTest and QueryWorkerTest.

  • Mutation check: I broke 14 of the riskiest changed lines one at a time, each in a scratch worktree. Every change made at least one named test fail. The lines covered:

    • error frame without a position, and decoding with the wrong request flags
    • the bit-63 varint rejection
    • dropping detail flags for servers without the capability, and re-checking capabilities after failover
    • the refusal of skipRows/maxRows on an older server
    • skip_rows before max_rows on the wire
    • row_count before stats in RESULT_END
    • pooled end-of-query events
    • malformed RESULT_END treated as a transport error
    • skipRows > 0 setting its flag
    • client-side errors going to the 4-argument onError
    • the error position reaching QueryException
    • the facade using the options copied at submit()

    One reordering in QueryImpl.submit() survives: copying the options after dispatch() instead of before. It doesn't matter in practice. The copy still finishes inside submit(), and no reasonable test hook could catch it. The realistic regression, the worker reading the handle's live options, is caught by testOptionsSetWhileInFlightApplyToTheNextSubmit.

  • Timing check: testDeadlineExpiredBeforeBackoffReportsNoPosition relies on a 2 s deadline. The window it guards takes 2–3 ms normally and at most 9 ms with every CPU core oversubscribed 2x. All runs passed, so I see no flake risk.

  • OSS tandem, feat(qwp): add error positions, timings, row counts and paging to QWP queries questdb#8356:

    • It uses the matching branch ia_query_info, and the two PRs link each other.
    • The client and server constants match. The trailer layouts also match: skip_rows then max_rows; row_count then the stats block; an i32 position.
    • QwpEgressQueryDetailsClientTest drives this client against a real server.
    • At dc0bff0a4e, which pins this exact client head, every OSS CI job passed, including enterprise-ci-light. The current tandem head 5610d9f767 is a master merge with the same pin, and its CI is still running.
  • Enterprise tandem: not needed.

    • The failover-loop changes only route client-side errors to the 4-argument onError and re-check server capabilities on each attempt. Local tests run the real failover loop against two mock servers.
    • Enterprise uses the OSS egress processor, which advertises CAP_QUERY_DETAILS for every node.
    • The store-and-forward drainer, pool startup, lazy_connect and role gating are untouched.

Summary

  • Gates: no Critical findings; the test gate passes with no admitted gaps.
  • Findings: 0 Critical, 0 Moderate, 0 Minor, so nothing in the diff or at callers outside it.
  • Callers checked across this repo, questdb (master and tandem branch) and questdb-ent:
    • Every handler implementation, including the Enterprise e2e sidecar, still gets exactly one terminal callback through the default overloads.
    • QueryLease is the only Query implementer.
    • Only QwpQueryClient calls submitQuery.
    • No existing execute(...) call changes overload or becomes ambiguous.
  • Submodules and binaries: no submodule pointer moved, and no binaries were committed.
  • CI at head: green, including the JDK 8 build and tests on x64 and ARM64, the JDK 25/26 compile jobs, and Azure on Linux, macOS and Windows.
  • Deliberate, documented tradeoffs:
    • Older servers: a query with skipRows/maxRows fails on the client before it is sent. Row count, stats and error position are silently left out. This also happens after a failover to an older node during a rolling upgrade, after the handler has seen onFailoverReset.
    • Reused details: the QwpQueryDetails passed to a handler is only valid during the callback.
    • Message format: with errorPosition(true), error messages arrive without the [n] prefix.
    • Missing trailer: a RESULT_END/EXEC_DONE without a requested trailer is treated as a transport error and can trigger failover. The tandem server always writes the trailers.
    • API change: Query gains an abstract options(). PR feat(qwp): connect timeout, ingest callbacks, and lazy_connect (tolerant startup) on the QuestDB facade #60 did the same with serverInfo() in 1.3.12 without flagging it as breaking.
  • Conventions: the title, body, labels, commit messages, member ordering, LOG usage (none added) and the Java 8 floor all follow them.

@glasstiger glasstiger added READY and removed tandem labels Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request QWP READY

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants