Leading comment support added for AlterTable, CreateTable, and ColumnDef 
          #2069
        
          
      
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
This PR refactors the internal representation of comments in the tokenizer and parser and adds leading comment support for
AlterTable,CreateTable, andColumnDef.The
SingleLineandMultiLinecomment variants previously defined in theWhitespaceenum are now represented by a new, dedicatedCommentenum. TheCommentis used for both whitespace comments (interstitial comments) and leading comments. Leading comment support propagated forAlterTable,CreateTable, andColumnDef, to parse leading comments alongside the associatied struct (see below for example).Rationale
This change improves the semantic clarity of comment handling in the SQL tokenizer and parser.
As discussed in #2065, comments preceding a table or column definition may serve as inline documentation, and should be distinguishable from interstitial (whitespace) comments.
For example:
Term Definitions
Intersitial Comment: a comment preceded by something (if nothing then defined as leading comment) that is not a comma or semicolon:
Leading Comment: a comment that is preceded by either nothing, a comma, or a semicolon.
currently the variants covered include single line comments:
and multi-line comments:
By separating comment handling from generic whitespace, the parser can now support more context-aware interpretations and contribute to a lossless syntax tree, addressing #175 and complementing PR #189.
Summary of Changes
Commentenum encapsulatingSingleLineandMultiLinecomment variants.Whitespaceto includeInterstitialComment(Comment)variant.Tokenizer.rsto emitCommentvalues instead ofWhitespace::SingleLineComment/Whitespace::MultiLineComment.parser/mod.rsimplementedLeadingCommentforCreateTable,ColumnDef,AlterTableparsing.leading_comment: Option<Comment>to theCreateTable,ColumnDef,AlterTable:InterstitialCommentandLeadingComment.