Skip to content

fix: block inverted index V1 creation in FE [DORIS-26415]#64522

Open
LIANG751234313 wants to merge 1 commit into
apache:masterfrom
LIANG751234313:fix/DORIS-26415-block-inverted-index-v1
Open

fix: block inverted index V1 creation in FE [DORIS-26415]#64522
LIANG751234313 wants to merge 1 commit into
apache:masterfrom
LIANG751234313:fix/DORIS-26415-block-inverted-index-v1

Conversation

@LIANG751234313

Copy link
Copy Markdown

Proposed changes

Issue Number: close #DORIS-26415

Problem

When users copy historical DDL statements that explicitly set inverted index format to V1, new tables are silently created with V1. Inverted index V1 does not support small file merging and bypasses the packed file path, significantly amplifying PUT requests to object storage.

Observed impact: clusters with high write QPS are seeing300–400 PUT/s, with 70–80% attributed to inverted index V1.

Solution

Block new inverted index V1 creation at the FE level:

  • Explicitly specifying inverted_index_storage_format = v1 on a new indexnow returns an error immediately.
  • Users are guided to use V2 (the default).
  • Existing V1 tables remain fully compatible — metadata loading and read/write
    are unaffected. No forced migration.

Error message:Inverted index V1 is deprecated and no longer allowed for new index creation.Please use inverted index V2.

Coverage

  • CREATE TABLE with inline index definition
  • ALTER TABLE ADD INDEX
  • All other code paths that create a new inverted index

Checklist

  • Does it affect the existing behavior? No — only blocks new V1 creation;existing V1 tables are untouched.
  • V2 is still used by default when the version is not specified.
  • Has unit/regression test been added? Yes — FE regression test added.

Explicitly specifying inverted index V1 format now returns an error to
prevent new tables from using V1. V1 does not support small file merging
and bypasses the packed file path, causing excessive PUT requests to
object storage (~300-400/s observed, 70-80% from inverted V1).

Error: Inverted index V1 is deprecated and no longer allowed for new
index creation. Please use inverted index V2.

Existing V1 tables remain compatible, no forced migration.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants