Describe the bug
Random Row fails for DECIMAL primary keys with 30 digits. The error is likely because these are being cast into floats, and the no rows match due the loss of precision
Steps to reproduce the behavior
See that the table has proper data with a large decimal number as primary key.
data-validation query -c postgres -q 'select id, col_data from pso_data_validator.dvt_large_decimals'
[(Decimal('123456789012345678901234567890'), 'Row 1'), (Decimal('223456789012345678901234567890'), 'Row 2'), (Decimal('323456789012345678901234567890'), 'Row 3'), (Decimal('423456789012345678901234567890'), 'Row 4'), (Decimal('523456789012345678901234567890'), 'Row 5')]
Run validation on this data
data-validation validate row -tc postgres -sc postgres -tbls=pso_data_validator.dvt_large_decimals -hash=col_data -rr -rbs=3
╒═══════════════════╤═══════════════════╤═════════════════════╤══════════════════════╤════════════════════╤════════════════════╤══════════════════╤═════════════════════╤══════════╕
│ validation_name │ validation_type │ source_table_name │ source_column_name │ source_agg_value │ target_agg_value │ pct_difference │ validation_status │ run_id │
╞═══════════════════╪═══════════════════╪═════════════════════╪══════════════════════╪════════════════════╪════════════════════╪══════════════════╪═════════════════════╪══════════╡
╘═══════════════════╧═══════════════════╧═════════════════════╧══════════════════════╧════════════════════╧════════════════════╧══════════════════╧═════════════════════╧══════════╛
This should show 3 rows and is showing 0 rows, likely because of the loss of precision in converting decimal to float. Also see random_row_validation.md for an alternative approach to random row validation.
Sundar Mudupalli
Describe the bug
Random Row fails for DECIMAL primary keys with 30 digits. The error is likely because these are being cast into floats, and the no rows match due the loss of precision
Steps to reproduce the behavior
See that the table has proper data with a large decimal number as primary key.
Run validation on this data
This should show 3 rows and is showing 0 rows, likely because of the loss of precision in converting decimal to float. Also see random_row_validation.md for an alternative approach to random row validation.
Sundar Mudupalli