Wednesday, March 7, 2012

Erlang's open_port and buffering

I've just spent 3 hours trying to figure out why open_port isn't working.
I had this command that I would spawn that would output a line of text from time to time. What I wanted to do is turn those lines of text into some form of events, in Erlang. Something like:

  run() ->
    Port = open_port({spawn, "my-long-running-command"},
                     [exit_status, {line, 1024}]),

  loop() ->
    receive {data, {eol, Line}}} -> we_have_an_event(Line) end,

But the problem was that Erlang was not getting anything from that command - even though it was printing when testing it separately.

So I spent a lot of time trying all the possible combinations of options for open_port - thinking that I'm not using it correctly.

At some point I discovered a curious thing. If I did "my-long-running-command | tee a.txt" nothing would show up in a.txt when running the command in open_port - but it would if I ran it in a terminal.

So it finally struck me. The command's output was being buffered somehow only when it was run in Erlang.
So the solution was to make all the buffers zero, like so:

  Port = open_port({spawn, "stdbuf -i0 -o0 -e0 my-long-running-command"},
                   [exit_status, {line, 1024}])

This way I received each line the moment it was printed out by the program.

No comments:

Post a Comment