Friday, June 8, 2012

How do I write stderr to a file while using "tee' with a pipe?


I have the below command line argument which will print the output of aaa.sh to the screen while also writing stdout to bbb.out; however I would also like to write stderr to a file ccc.out. Any suggestions on how to modify the below piece? Thanks!




./aaa.sh | tee ./bbb.out



Update: stdout and stderr should still both be printed to the screen, regardless.


Source: Tips4all

4 comments:

  1. I'm assuming you want to still see STDERR and STDOUT on the terminal. You could go for Josh Kelley's answer, but I find keeping a tail around in the background which outputs your log file very hackish and cludgy. Notice how you need to keep an exra FD and do cleanup afterward by killing it and technically should be doing that in a trap '...' EXIT.

    There is a better way to do this, and you've already discovered it: tee.

    Only, instead of just using it for your stdout, have a tee for stdout and one for stderr. How will you accomplish this? Process substitution and file redirection:

    command > >(tee stdout.log) 2> >(tee stderr.log >&2)


    Let's split it up and explain:

    > >(..)


    >(...) (process substitution) creates a FIFO and lets tee listen on it. Then, it uses > (file redirection) to redirect the STDOUT of command to the FIFO that your first tee is listening on.

    Same thing for the second:

    2> >(tee stderr.log >&2)


    We use process substitution again to make a tee process that reads from STDIN and dumps it into stderr.log. tee outputs its input back on STDOUT, but since its input is our STDERR, we want to redirect tee's STDOUT to our STDERR again. Then we use file redirection to redirect command's STDERR to the FIFO's input (tee's STDIN).

    See http://mywiki.wooledge.org/BashGuide/InputAndOutput

    Process substitution is one of those really lovely things you get as a bonus of choosing bash as your shell as opposed to sh (POSIX or Bourne). In sh, you'd have to do things manually:

    out="${TMPDIR:-/tmp}/out.$$" err="${TMPDIR:-/tmp}/err.$$"
    mkfifo "$out" "$err"
    trap 'rm "$out" "$err"' EXIT
    tee stdout.log < "$out" &
    tee stderr.log < "$err" >&2 &
    command >"$out" 2>"$err"

    ReplyDelete
  2. why not simply:

    ./aaa.sh 2>&1 | tee -a log


    This simply redirects stdout to stderr, so tee echoes both to log and to screen. Maybe I'm missing something, because some of the other solutions seem really complicated.

    ReplyDelete
  3. To redirect stderr to a file, display stdout to screen, and also save stdout to a file:

    ./aaa.sh 2>ccc.out | tee ./bbb.out

    EDIT: To display both stderr and stdout to screen and also save both to a file, you can use bash's I/O redirection:

    #!/bin/bash

    # Create a new file descriptor 4, pointed at the file
    # which will receive stderr.
    exec 4<>ccc.out

    # Also print the contents of this file to screen.
    tail -f ccc.out &

    # Run the command; tee stdout as normal, and send stderr
    # to our file descriptor 4.
    ./aaa.sh 2>&4 | tee bbb.out

    # Clean up: Close file descriptor 4 and kill tail -f.
    exec 4>&-
    kill %1

    ReplyDelete
  4. If using bash:

    # Redirect standard out and standard error separately
    % cmd >stdout-redirect 2>stderr-redirect

    # Redirect standard error and out together
    % cmd >stdout-redirect 2>&1

    # Merge standard error with standard out and pipe
    % cmd 2>&1 |cmd2


    Credit (not answering from the top of my head) goes here: http://www.cygwin.com/ml/cygwin/2003-06/msg00772.html

    ReplyDelete