You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Enhancement] implement streamload via http stream #36773
I had searched in the issues and found no similar issues.
Description
Currently we can put data to doris via http using streamload or http stream. httpstream transforms data via sql in a header while streamload via special header. Sql is more easier to use.
In future, we would suggest http stream for users, and we also need keep only one in internal implementation. So we can add a config option to enable implement streamload via ways of httpstream.
We can transform streamload headers to a sql internally and implement streamload in ways of httstream.
Solution
In the streamload action of be, collector headers and transfrom it to sql.
I'm working on this issue, but I got some problem in parsing stream load headers with "columns" set in header.
For example, for a table with columns (c1,c2,c3) and an stream load request has 'columns' = "c1, c2, c3,temp" in request header.
If I convert this into the form[ insert into xxx.xx select * from http_stream("columns"="c1, c2, c3,temp") ], I get an insert into cols should be corresponding to the query output error.
It seems the process steps in httpstream requires the input cols equal to table columns. However, the streamLoad opeation support the input cols size more than table columns.
I wonder what should I do to solve this problem. Thanks a lot.
We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!
Activity
Tech-Circle-48 commentedon Jun 24, 2024
please assign to me
StarryVerse commentedon Oct 21, 2024
Has this issue been resolved? Please assign it to me
[feature](stream-load)(config) implement streamload via http stream (a…
[Enhancement](stream-load)(config) implement streamload via http stre…
[Feature](stream-load)(config) implement streamload via http stream (a…
[Feature](stream-load)(config) implement streamload via http stream (a…
[Feature](stream-load)(config) implement streamload via http stream (a…
[Feature](stream-load)(config) implement streamload via http stream (a…
[Feature](stream-load)(config) implement streamload via http stream (a…
[Feature](stream-load)(config) implement streamload via http stream (a…
[Feature](stream-load)(config) implement streamload via http stream (a…
Yufei-YAO commentedon Nov 11, 2024
I'm working on this issue, but I got some problem in parsing stream load headers with "columns" set in header.
For example, for a table with columns (c1,c2,c3) and an stream load request has 'columns' = "c1, c2, c3,temp" in request header.
If I convert this into the form[ insert into xxx.xx select * from http_stream("columns"="c1, c2, c3,temp") ], I get an insert into cols should be corresponding to the query output error.
It seems the process steps in httpstream requires the input cols equal to table columns. However, the streamLoad opeation support the input cols size more than table columns.
I wonder what should I do to solve this problem. Thanks a lot.
[feature](stream-load)(config) implement streamload via http stream (a…
github-actions commentedon Sep 12, 2025
We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!