Changelog
Release history for the DLH.io SQL Server Agent: new capabilities, improvements, fixes and security changes by version.
Every agent release is listed here, newest first. The version matches agent_version in the configuration template shipped with that release, so dlh_agent_sql_server.exe --version and --config-check tell you exactly which entry applies to your installation and whether your configuration file is behind.
Upgrading
Read the entries between your installed version and the latest one, then follow the upgrade steps in Operations. New configuration keys are always optional with backwards-compatible defaults; --config-check lists anything that changed.
Observability, schema drift remediation and SCD Type 2
The hardening release for unattended fleets: every run now explains itself (tier banner, state store telemetry, run summary), a read-only --diagnose command checks a host before anything is extracted, incompatible column changes are detected and remediated automatically for Delta and Iceberg, and SCD Type 2 history is available as an opt-in write strategy.
- dlh_agent_sql_server.exe --diagnose: read-only host health check covering configuration, credentials, ODBC drivers, SQL Server and CT/CDC, platform bootstrap, storage, the DuckDB state store, disk and memory, with PASS/WARN/FAIL per check, --json output for support and --offline mode for hosts without outbound access.
- State store telemetry block at the top of every run log: state file path and size, created and modified time, tables tracked by sync mode, run history counts, last successful run, tables whose last run failed and pending schema drift.
- SCD Type 2 write strategy (write_strategy: scd2) for Delta Lake and Apache Iceberg targets, tracking row history with _scd_valid_from, _scd_valid_to and _scd_is_current. Opt-in per table or for a whole database.
- Database-level write strategy: data_retrieval.write_strategy sets the default for every table and per-table write_strategy overrides it. Unset configurations keep the previous merge behaviour.
- Schema drift handling for Apache Iceberg targets, matching the Delta Lake behaviour (detect, fail_once, coerce_to_target, migrate_target).
- TRIAL countdown in the tier banner ("N day(s) remaining - expires YYYY-MM-DD") with a warning notification when the remaining days reach dlh_ref.trial_expiry_warning_days (default 7).
- Optional dlh_log_store: when platform credentials are available, remote log shipping uses them and the section can be omitted from the configuration.
- New Delta Lake and Iceberg tables type decimal, numeric, money and smallmoney columns from the SQL Server declared precision and scale instead of the values in the first batch, so later larger values no longer fail with "too large to store in a Decimal128".
- Change Tracking batches where every changed row has NULL in a column now keep the target column type instead of failing the merge with a type coercion error.
- Schema drift errors name the affected columns with source and target types, and the drift action is applied to that table only on the next run.
- When a table is configured for scd2 but has no primary key the agent logs a warning and uses merge for that table instead of failing the run.
- Platform bootstrap resolves storage credentials issued under earlier field names, supports static Azure account keys, and prints redacted storage diagnostics when validation fails.
- Platform run reports send run_failed and run_warning events so the DLH.io platform can distinguish a failed run from one that completed with table errors.
- The agent recreates the base credentials file when it is missing and offers to re-enter SQL Server credentials when the encrypted companion cannot be read.
- Windows executable hardening: the packaged build no longer contains an extractable bytecode archive, and compression that interfered with anti-virus scanners was removed.
Platform-issued storage credentials and Community mode
Storage credentials can now be issued by the DLH.io platform at run time, so the configuration file no longer needs to contain keys or SAS tokens. Runs without a valid API key continue in Community mode.
- dlh_storage is optional when dlh_ref.dlh_api_key is set: storage credentials are obtained from the DLH.io platform bootstrap, cached encrypted on the host and refreshed automatically.
- Community mode: a configuration without a valid API key or storage runs locally with a 10,000 row per table cap and no cloud upload, so the agent can be evaluated before a connection is provisioned.
- Tier and license banner printed after bootstrap with tier, connection, storage provider, credential source, row cap and cloud upload status.
- DLH.io boot banner at the top of every run with version, run id, host, configuration file, log file and connection identifiers.
- --clear-credentials-cache CLI flag to remove the encrypted bootstrap cache and force a live validation on the next run.
Run report payload limits
Run reports sent to the DLH.io platform stay small on very large estates and link back to the shipped logs.
- Run report payloads cap the per-table list at 50 entries and add tables_total and tables_truncated so summaries remain compact for connections with thousands of tables.
- Run reports include logs_location, the object storage URL of the run folder, so the full log can be opened from the platform.
dbdeux scheduled jobs after a run
Trigger dbdeux (dbtx) scheduled jobs as a post-completion action so transformations start as soon as fresh data lands.
- post_completion_actions.dbdeux_jobs with run_condition (on_success, on_error, always), variables and schedule_window support.
- Named credentials of type api for dbdeux service-account tokens, encrypted on first run like SCM tokens.
Platform run reporting
Every run reports to the DLH.io platform so agents can be watched centrally and silent hosts are noticed.
- dlh_notifications.platform_run_report (default true): the agent posts run_started and run_completed events with a per-table summary to the DLH.io platform.
Run-keyed remote logs
Shipped logs are organised per run, redacted, and accompanied by a machine-readable summary.
- Remote logs are stored under logs/<YYYY-MM-DD>/run_<RUN_ID>/ together with a run_<RUN_ID>_summary.json sidecar.
- Logs are uploaded on fatal errors as well as successful runs.
- Passwords, API keys, SAS tokens and webhook URLs are scrubbed from logs before they are shipped.
Slack, Teams and webhook alerts
Failure and warning alerts can go to chat channels and incident tools, not only email.
- dlh_notifications.webhooks: Slack, Microsoft Teams (Adaptive Card) and generic JSON webhook channels with per-webhook format, error_severity override and url_env_var for keeping the URL out of the configuration file.
- dlh_notifications.webhook_timeout_seconds.
- Schema drift detection for Delta Lake targets with on_schema_drift_action (fail, coerce_to_target, migrate_target) and on_schema_drift_action_handling (do_nothing, fail_once, do_schema_drift_action).
Safer Delta vacuum defaults
Default retention now protects external readers that replay the Delta log incrementally.
- delta_vacuum_retention_hours default changed from 0 to 168 (7 days). Snowflake Iceberg REFRESH and similar readers need recently superseded files during log replay.
- Startup warning when delta_vacuum_retention_hours is below 168 with direct_cloud_write enabled.
Parquet compression options
Choose the Parquet codec used inside Delta Lake and Iceberg tables.
- parquet_compression (snappy, zstd, gzip, uncompressed) for Delta Lake and Iceberg output. Default snappy.
- parquet_compression_level for codecs that support levels, for example ZSTD 1 to 22.
Lifecycle actions
Run platform and CI/CD work before and after each agent run.
- lifecycle_actions with pre_start_actions and post_completion_actions: Sync Bridge calls and CI/CD dispatches to GitHub, GitLab and Bitbucket with provider, owner, repo, event_type, branch, dbt_profile_target, workflow_ref and variables.
- dlh_ref.api_key is encrypted on first run.
- workflow_ref is optional (GitLab and Bitbucket only); dbt_profile_target and branch are included in dispatch payloads.
Iceberg row-level deletes
Deletes captured by Change Tracking or CDC are applied to Iceberg tables.
- Row-level delete support for Apache Iceberg targets.
- sql_queries.output_path for named cloud directories.
- sql_queries output_path renamed to output_folder_prefix for local output.
Azure URI schemes and Iceberg metadata controls
More control over how Delta Lake paths are written for Azure and how Iceberg metadata is retained.
- azure_storage_uri_scheme (az, abfss, wasbs, azure) for Delta Lake cloud paths.
- iceberg_cloud_warehouse_uri, iceberg_version_hint, iceberg_metadata_retention and iceberg_path_style.
Retention settings
Configurable retention for Delta history, agent logs and the audit trail.
- delta_vacuum_retention_hours and delta_log_retention_duration.
- agent_log_retention_days and audit_trail_retention_days.
Output layout and naming
Control how databases, schemas and tables map to folders and table names in the target.
- table_name_structure (flat, nested, db_combined, db_separated) for Delta Lake and Iceberg output hierarchy.
- table_name_casing (default, uppercase, lowercase).
- sql_queries (formerly sql_files) with database scoping and per-file output format.
- create_delta_table_crc for CRC checksum files.
Initial versioned release
First release with a versioned configuration template.
- Full, Change Tracking and CDC synchronization of SQL Server tables and views to CSV, Parquet, Delta Lake and Apache Iceberg.
- connection_information, data_retrieval, dlh_storage, dlh_log_store, dlh_notifications, produce_audit_trail, direct_cloud_write, historical_load, state_db_path and odbc_driver configuration sections.
- Encrypted SQL Server credentials and a local DuckDB state store for incremental sync pointers.
Release notes for the wider platform
Agent releases that affect the DLH.io platform (run reporting, remote log viewing, connection provisioning) are also summarized on the DLH.io changelog.
Requesting a release
To download the latest installer or ask for a feature, contact us.