Skip to content

stream load 用json导入时发生内存泄漏 #4962

Closed

Description

@PeterGZW

服务器配置 1fe3be , 8核16G ,内存都是默认参数

1.每次查询mysql 1w条记录 转成json文件
2.通过curl 导入进doris。平均doris耗时1s消费一批json。 (单个json文件大小在20-40M。)
3.稳定跑1h左右,3台内存有一台会达到90%以上,然后突然无响应。过一段时间后进程被系统kill,发生了OOM。

版本doris 0.13.11-release

Activity

  1. Dam1029 commented on Dec 9, 2020

    @Dam1029
    Contributor

    顶一下,这边也遇到了类似的问题,我们使用flink封装了stream load,并将json文件实时发送到doris后,be的内存值doris_be_memory_allocated_bytes一直上涨到OOM,即便任务停止,内存监控值也下不来(没有释放),重启才能恢复

  2. stdpain commented on Dec 12, 2020

    @stdpain
    Contributor

    issues #5069
    pull request #5073

  3. LKK1535977706 commented on Jul 5, 2023

    @LKK1535977706

    @Dam1029 请问这个问题解决了吗,我这边也遇到同样的问题了

  4. dataroaring commented on Sep 3, 2023

    @dataroaring
    Contributor

    Which version do you use?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions