Highlights
This release brings a few minor breaking changes, both for the core project and for the dbt templater. For the main project:
-
Resolving an issue with the spacing of functions (LT01), which involved a change to how functions are parsed. If your project relies on the specific parsing of functions, the bracketed arguments are now wrapped in a
function_contents
object. We recommend that you examine the new parsing structure using this new release in testing first. -
RF06
(references.quoting
) is now case sensitive when removing quotes which are detected as unnecessary. This rule has also been re-enabled by default for Snowflake and Postgres where it had previously been disabled (for the reason that in the past it hadn’t been appropriately case sensitive). Treatment for totally case-insensitive dialects like DuckDB and SparkSQL have also been included. Please check the new documentation for this rule (which is much more explicit now), for details related to your dialect. -
Patterns equivalent to those from
.sqlfluffignore
can now be included in.sqlfluff
andpyproject.toml
files. -
Using the
python
templater, users now have an option to include variables which include a dot in the path, like{{ foo.bar }}
using a specialsqlfluff
context variable. -
Significant changes under the hood to the handling of configuration files. Most of these should not be visible to end users, but for anyone integrating SQLFluff into a larger project and relying on native file loading may need to refactor their project for this release. Most notably here, for maintainers of plugins, the
ConfigLoader
class has been deprecated, and plugins should instead call the config loading functions directly. See the example plugin for details. -
Documentation, especially for dialects, has been significantly improved. Documentation for
CP02
(capitalisation.identifiers
) has also been clarified to make it’s implication for references and aliases more clear. -
During testing, to isolate the effect of specific rules, there’s a new CLI option
--disable-noqa-except
which allows allnoqa
options to be ignored except the ones provided in this option.
For the dbt templater:
-
Support for dbt 1.1-1.3 has been removed. All have been in End of Life (EOL) support by dbtlabs for almost two years. They are also poorly supported by other projects and tools.
-
The dbt templater has been migrated to use
pyproject.toml
. -
Handling of errors and exceptions raised within dbt has had an overhaul. Users may see a slightly different presentation of errors, but the overall stability should be more robust.
In addition to those changes, there have been too many dialect contributions and bugfixes to mention specifically. We’ve also seen six people make their first contributions to the project as part of preparing for this release! 🎉🏆🎉.
What’s Changed
- Handle multi-processing dbt exceptions much better. (#6138) @alanmcruickshank
- Support variables with dot in python templater (#5872) @timchurch
- Add postgres normalization operator support (#6211) @fnimick
- Fix patch will anchor on first buffer insertion point (#6212) @keraion
- Allow ignore patterns in other config files. (#6130) @alanmcruickshank
- Strict typing in
config
andhelpers
. (#6206) @alanmcruickshank - TSQL: Support multiple options in
SET
statement (#6205) @keraion - DuckDB: Support
CREATE TYPE
statement (#6204) @keraion - Update Slack link (#6203) @greg-finley
- Add quoted literal checking for Snowflake TARGET_LAG in dynamic tables. (#6201) @mvastarelli
- Databricks: Support
COMMENT ON
statement (#6196) @keraion - DuckDB: Support
STRUCT
datatype (#6198) @keraion - Deprecate the
ConfigLoader
(#6177) @alanmcruickshank - DuckDB: Support
CREATE MACRO
/CREATE FUNCTION
(#6194) @keraion - DuckDB: Support functions with walrus operators (#6193) @keraion
- Add volume syntax support for Databricks (#6179) @TheCleric
- Handle errors better in AL09 (#6186) @alanmcruickshank
- Add support for managed locations to databricks dialect schemas (#6182) @TheCleric
- MYSQL: Create Table Optional AS (#6109) @WittierDinosaur
- More dialect documentation (#6165) @alanmcruickshank
- Better documentation on how to cross reference rules and fix a few. (#6162) @alanmcruickshank
- RF06: Case Sensitivity (#6173) @alanmcruickshank
- SparkSQL/Databricks: Support for
VARIANT
type (#6167) @keraion - sparksql: Allow
INSERT OVERWRITE
after a CTE (#6172) @keraion - postgres: Add
SET CONSTRAINTS
statement (#6171) @keraion - TSQL: Fix
MERGE
without a target alias (#6170) @keraion - TSQL: add
OFFSET
andFETCH
(#6169) @keraion - postgres: Add support for
SUBSCRIPTION
statements (#6168) @keraion - Duckdb: Add support for list comprehensions (#6166) @keraion
- Update Docs and tests for CP02 (#6163) @alanmcruickshank
- Cached property in RF06 rather than DIY (#6164) @alanmcruickshank
- CI: Update
util.py
for dbt templaterpyproject.toml
(#6160) @keraion - Auto generate dialect docs (#6153) @alanmcruickshank
- Deprecate support for dbt 1.1-1.3 (#6159) @WittierDinosaur
- ST08: Ignore
DISTINCT
s with subqueries (#6146) @keraion - Duckdb: Fix Create View coverage (#6158) @WittierDinosaur
- Snowflake: Support Password Policies (#6154) @WittierDinosaur
- Postgres: Allow negative integers in sequences (#6111) @WittierDinosaur
- Postgres: Add SHOW Statement (#6110) @WittierDinosaur
- Migrate dbt templater to pyproject.toml (#6155) @keraion
- Snowflake: Add DEFAULT option for function parameters (#6145) @keraion
- Snowflake: fixes parsing for INCLUDE_METADATA in COPY INTO statement (#6150) @jcrobak
- [SNOWFLAKE] Adding support for extended constraint definitions (#6151) @babak-l1
- Snowflake: fixes parsing for PARSE_HEADER in FILE FORMAT statement (#6149) @jcrobak
- fix: avoid strip_newlines when encounter comments in inline segments (#6140) @Cynthia-Cheng
- More robust exception handling for dbt. (#6144) @alanmcruickshank
- postgres: Add
ENCRYPTED PASSWORD
option inCREATE USER
(#6143) @keraion - Fix support of INTERVAL in ClickHouse (#6112) @Pavel-Strybuk
- Add support for Snowflake Higher-Order Functions (#6136) @amardatar
- Method extraction and more robust typing in config. (#6135) @alanmcruickshank
- Add missing databricks and sparksql ALTER statements (#6102) @pahunter90
- fix: program_counter move in JinjaTracer (#6121) (#6123) @Cynthia-Cheng
- CI: allow hidden file upload for coverage files (#6139) @keraion
- Fix: added DOWNSTREAM keyword for TARGET_LAG on dynamic tables in Snowflake. (#6131) @mvastarelli
- Trino Dialect: update ARRAY type handling (#6127) @kirkhansen
- Split apart config module (#6128) @alanmcruickshank
- Add option for allowing only a subset of rules for noqa (#6115) @keraion
- TSQL: Allow for empty catch block in try-catch (#6116) @keraion
- Change evaluation order of literals before column refs in bracketed, delimited expressions (#6117) @keraion
- Fix spacing rules for functions (#5809) @WittierDinosaur
- SQLite: Add conflict_clause to unique table constraint (#6106) @WittierDinosaur
- SQLite: Support Raise Function (#6108) @WittierDinosaur
- SQLite: Create Trigger WHEN optionally bracketed (#6107) @WittierDinosaur
- Snowflake: Added
INTERVAL
s to Frame Clause (#6105) @keraion - Postgres: Add
IS UNKNOWN
(#6094) @keraion - RF02: Handle subquery column qualification (#6091) @keraion
- tsql: Allow leading dots in table references (#6093) @keraion
Source: https://github.com/sqlfluff/sqlfluff/releases/tag/3.2.0