Description
Running row-level validations against SQL Server with --random-row-batch-size > 2000 on a table containing more than ~2,000 rows, execution fails with pyodbc.Error SQLSTATE 07002:
(pyodbc.Error) ('07002', '[07002] [Microsoft][ODBC Driver 18 for SQL Server]COUNT field incorrect or syntax error (0) (SQLExecDirectW)')
[SQL: WITH t0 AS
(SELECT t6.id AS id, t6.col1 AS col1
FROM schema.table_name AS t6
WHERE t6.id IN (?, ?, ..., ?))
...
[parameters: (100343, 568208, ... 4902 parameters truncated ... 'DEFAULT_REPLACEMENT_STRING', 'DEFAULT_REPLACEMENT_STRING')]
In ODBC terminology, the COUNT field in SQLSTATE 07002 is the Application Parameter Descriptor (SQL_DESC_COUNT) header field — indicating that the number of bound ? parameters in SQLExecDirectW exceeded SQL Server's hard 2,100 parameter limit per statement.
Potential solutions
- For SQL Server with random row sampling we could cap the effective random-row batch size in
_add_random_row_filter() / ConfigManager so the total bound parameters per statement cannot exceed 2,100:
max_mssql_rows = max(1, (2000 - num_hashed_columns) // len(source_pk_columns))
effective_batch_size = min(batch_size, max_mssql_rows)
-
Alternatively (or in addition), we could inline literal values in IN (...) predicates on instead of emitting SQLAlchemy bound ? parameters. If we decided to do this we would need to ensure we block SQL injection risks.
-
Output a warning and leave it to the user to provide a batch size appropriate to the table being validated.
Description
Running row-level validations against SQL Server with
--random-row-batch-size > 2000on a table containing more than ~2,000 rows, execution fails withpyodbc.ErrorSQLSTATE07002:In ODBC terminology, the COUNT field in SQLSTATE 07002 is the Application Parameter Descriptor (SQL_DESC_COUNT) header field — indicating that the number of bound ? parameters in SQLExecDirectW exceeded SQL Server's hard 2,100 parameter limit per statement.
Potential solutions
_add_random_row_filter()/ConfigManagerso the total bound parameters per statement cannot exceed 2,100:Alternatively (or in addition), we could inline literal values in IN (...) predicates on instead of emitting SQLAlchemy bound ? parameters. If we decided to do this we would need to ensure we block SQL injection risks.
Output a warning and leave it to the user to provide a batch size appropriate to the table being validated.