Skip to content

generate-table-partitions make error #1705

Description

@Threeboys33

Describe the bug

The YAML file generated using the generate-table-partitions method contains an SQL statement with the :param_1 parameter. When subsequently running config run, the following error occurs:
parse error: Unfinished string at EOF at line 1, column 248180736

What version of DVT are you using?
DVT: 8.5.2, python 3.11

What type of connections are you using for source and target?
Ex: Mysql 8.x for source, PG 9.x for target

Steps to reproduce the behavior

  1. Add 'data-validation generate-table-partitions -sc mysql -tc dws
    -pk "l_orderkey,l_partkey,l_suppkey,l_linenumber"
    --tables-list "test_db.lineitem=public.lineitem"
    --hash "l_shipinstruct"
    -cdir partions
    --partition-num 10
    --format json'
  2. Execute 'data-validation -v -ll DEBUG configs run -c partions/test_db.lineitem/0000.yaml | jq '.''
  3. Scroll down to '03/19/2026 07:27:29 PM-INFO: -- ** Source Query ** --
    03/19/2026 07:27:29 PM-INFO: SELECT t0.hash__all, t0.l_orderkey, t0.l_partkey, t0.l_suppkey, t0.l_linenumber
    FROM (SELECT t1.l_orderkey AS l_orderkey, t1.l_partkey AS l_partkey, t1.l_suppkey AS l_suppkey, t1.l_linenumber AS l_linenumber, t1.l_quantity AS l_quantity, t1.l_extendedprice AS l_extendedprice, t1.l_discount AS l_discount, t1.l_tax AS l_tax, t1.l_returnflag AS l_returnflag, t1.l_linestatus AS l_linestatus, t1.l_shipdate AS l_shipdate, t1.l_commitdate AS l_commitdate, t1.l_receiptdate AS l_receiptdate, t1.l_shipinstruct AS l_shipinstruct, t1.l_shipmode AS l_shipmode, t1.l_comment AS l_comment, t1.cast__l_shipinstruct AS cast__l_shipinstruct, t1.ifnull__cast__l_shipinstruct AS ifnull__cast__l_shipinstruct, t1.rstrip__ifnull__cast__l_shipinstruct AS rstrip__ifnull__cast__l_shipinstruct, t1.concat__all AS concat__all, sha2(t1.concat__all, '256') AS hash__all
    FROM (SELECT t2.l_orderkey AS l_orderkey, t2.l_partkey AS l_partkey, t2.l_suppkey AS l_suppkey, t2.l_linenumber AS l_linenumber, t2.l_quantity AS l_quantity, t2.l_extendedprice AS l_extendedprice, t2.l_discount AS l_discount, t2.l_tax AS l_tax, t2.l_returnflag AS l_returnflag, t2.l_linestatus AS l_linestatus, t2.l_shipdate AS l_shipdate, t2.l_commitdate AS l_commitdate, t2.l_receiptdate AS l_receiptdate, t2.l_shipinstruct AS l_shipinstruct, t2.l_shipmode AS l_shipmode, t2.l_comment AS l_comment, t2.cast__l_shipinstruct AS cast__l_shipinstruct, t2.ifnull__cast__l_shipinstruct AS ifnull__cast__l_shipinstruct, t2.rstrip__ifnull__cast__l_shipinstruct AS rstrip__ifnull__cast__l_shipinstruct, concat_ws(:param_1, t2.rstrip__ifnull__cast__l_shipinstruct) AS concat__all
    '
    1. See error '03/19/2026 07:28:03 PM-INFO: {"run_id": "8103a851-a491-458f-b6fa-77379a8591e1", "start_time": "2026-03-19T11:27:29.642633+00:00", "end_time": "2026-03-19T11:27:43.132156+00:00", "total_source_rows": 1199799, "total_target_rows": 1199799, "total_rows_validated": 1199799, "total_rows_success_validation_status": 1199799, "total_rows_fail_validation_status": 0, "failed_rows_present_in_source_not_in_target": 0, "failed_rows_present_in_target_not_in_source": 0, "failed_rows_present_in_both_source_and_target": 0}
      03/19/2026 07:28:03 PM-DEBUG: Generate report elapsed: 22.75s
      parse error: Unfinished string at EOF at line 1, column 248180736'
      Expected behavior
      A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

How are you executing DVT?
Ex: CLI commands on local computer or VM, within a DAG, Cloud Run job, etc

Additional context
Add any other context about the problem here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions