Skip to content

[feature wanted]现在Doris的union all好像是会集中到一台机器上进行计算,导致效率不高 #4656

Closed

Description

@li-ang-666

Doris版本:0.12.21

我有 N 个小SQL,每个SQL执行0.6秒左右

然后这N个SQL ,union all到一起,做一个group by,随着N的增大,从profile上来看,unionn_node的某个exchange_node的执行时间会越来越长

当N为20时,执行时间是24秒,最慢的这个exchange_node的activeTime与FirstBatchArrivalWaitTime均为23秒

通过询问dev,得出的结论可能是cpu切不过来了

是不是可以优化一下union all的代码逻辑,使其不要一次性发往一个节点?

还有一个想法,就是:我们集群目前是16核的CPU,是不是可以通过增加核心数,来提升当前union all逻辑下的性能?

Activity

  1. changed the title [-][union all]现在Doris的union all好像是会集中到一台机器上进行计算,导致效率不高[/-] [+][feature wanted]现在Doris的union all好像是会集中到一台机器上进行计算,导致效率不高[/+] on Sep 23, 2020
  2. xinghuayu007 commented on Sep 27, 2020

    @xinghuayu007
    Contributor

    能给一个复现的例子吗

  3. li-ang-666 commented on Oct 13, 2020

    @li-ang-666
    Author

    能给一个复现的例子吗

    我给超勇哥提供了例子~

  4. li-ang-666 commented on Oct 13, 2020

    @li-ang-666
    Author

    能给一个复现的例子吗

    @chaoyli 勇哥

  5. morningman commented on Nov 12, 2020

    @morningman
    Contributor

    There are two main problems here.

    1. The Plan process is inefficient.

    In the Plan phase, when multiple UNION combinations are encountered, multiple rounds of iteration will increase the plan time. In my test, it may cost 2-3 seconds to analyze and plan a SQL with 30 UNIONs.

    1. Too many Fragments

    A SQL may generate hundreds or thousands of Fragments, and all Fragments are currently sent to the corresponding BE separately. The common fields of these Fragments will undergo multiple serialization and deserialization operations, which results in a very high RPC processing time.
    In my test, a SQL with 30 UNIONs generates 300+ fragments, and total size of these fragments is over 100MB, mainly
    occupied by DescriptorTable.

    Optimization Strategy:

    1. For problem 1, it is necessary to analyze the corresponding code hot spots through the Java Profiler tool to optimize the analysis process. (TODO)
    2. For problem 2, multiple fragments sent to the same BE can be merged and sent in batch, reducing the number of RPCs and reducing the amount of data sent. (WIP)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions