Skip to content

[Feature](inverted index) Support double/float column type in bkd index - #17201

Closed
Tanya-W wants to merge 2 commits into
apache:masterfrom
Tanya-W:inverted-index
Closed

Tanya-W wants to merge 2 commits into
apache:masterfrom
Tanya-W:inverted-index

Conversation

@Tanya-W

@Tanya-W Tanya-W commented Feb 27, 2023

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

Problem summary

Support double/float column type in inverted index (for bkd index).

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions Bot added area/planner Issues or PRs related to the query planner area/vectorization kind/test labels Feb 27, 2023

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/olap/key_coder.h
public:
using CppType = typename CppTypeTraits<OLAP_FIELD_TYPE_FLOAT>::CppType;

public:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: redundant access specifier has the same accessibility as the previous access specifier [readability-redundant-access-specifiers]

Suggested change
public:

be/src/olap/key_coder.h:140: previously declared here

public:
^

Comment thread be/src/olap/key_coder.h
public:
static int64_t doubleToLong(double value) {
union UnionLongDouble {
int64_t long_;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: redundant access specifier has the same accessibility as the previous access specifier [readability-redundant-access-specifiers]

Suggested change
int64_t long_;

be/src/olap/key_coder.h:179: previously declared here

public:
^

@Tanya-W Tanya-W changed the title [Feature] (inverted index) Support double/float column type in bkd index [Feature](inverted index) Support double/float column type in bkd index Feb 27, 2023

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/olap/key_coder.h
public:
using CppType = typename CppTypeTraits<OLAP_FIELD_TYPE_DOUBLE>::CppType;

public:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: redundant access specifier has the same accessibility as the previous access specifier [readability-redundant-access-specifiers]

Suggested change
public:

be/src/olap/key_coder.h:176: previously declared here

public:
^

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/olap/key_coder.h
public:
using CppType = typename CppTypeTraits<OLAP_FIELD_TYPE_DOUBLE>::CppType;

public:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: redundant access specifier has the same accessibility as the previous access specifier [readability-redundant-access-specifiers]

Suggested change
public:

be/src/olap/key_coder.h:175: previously declared here

public:
^

@Tanya-W

Tanya-W commented Feb 28, 2023

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

hello-stephen commented Feb 28, 2023 •

Copy link
Copy Markdown
Contributor

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.61 seconds
stream load tsv: 480 seconds loaded 74807831229 Bytes, about 148 MB/s
stream load json: 40 seconds loaded 2358488459 Bytes, about 56 MB/s
stream load orc: 68 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 27 seconds loaded 861443392 Bytes, about 30 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230302091359_clickbench_pr_107063.html

@Tanya-W

Tanya-W commented Mar 2, 2023

Copy link
Copy Markdown
Contributor Author

run buildall

@Tanya-W

Tanya-W commented Mar 2, 2023

Copy link
Copy Markdown
Contributor Author

closed this pr first, because here are need to rethink the precision matching of float type equal query

@Tanya-W Tanya-W closed this Mar 2, 2023
@Tanya-W
Tanya-W deleted the inverted-index branch September 12, 2023 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner area/vectorization kind/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants