Friday, June 1, 2012

Best way to copy between two Stream instances - C#


What is the best way to copy the contents of one stream to another? Is there a standard utility method for this?



Source: Tips4all

10 comments:

  1. Feel free to edit this community post to fix bugs or implement improvements.

    public static void CopyStream(Stream input, Stream output)
    {
    byte[] buffer = new byte[32768];
    int read;
    while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
    {
    output.Write (buffer, 0, read);
    }
    }


    N.B. From .NET 4.0, there's a Stream.CopyTo method:

    input.CopyTo(output);


    And you have to set the position to 0 afterwards.

    input.Position = output.Position = 0;

    ReplyDelete
  2. I use the following extension methods. They have optimized overloads for when one stream is a MemoryStream.

    public static void CopyTo(this Stream src, Stream dest)
    {
    int size = (src.CanSeek) ? Math.Min((int)(src.Length - src.Position), 0x2000) : 0x2000;
    byte[] buffer = new byte[size];
    int n;
    do
    {
    n = src.Read(buffer, 0, buffer.Length);
    dest.Write(buffer, 0, n);
    } while (n != 0);
    }

    public static void CopyTo(this MemoryStream src, Stream dest)
    {
    dest.Write(src.GetBuffer(), (int)src.Position, (int)(src.Length - src.Position));
    }

    public static void CopyTo(this Stream src, MemoryStream dest)
    {
    if (src.CanSeek)
    {
    int pos = (int)dest.Position;
    int length = (int)(src.Length - src.Position) + pos;
    dest.SetLength(length);

    while(pos < length)
    pos += src.Read(dest.GetBuffer(), pos, length - pos);
    }
    else
    src.CopyTo((Stream)dest);
    }

    ReplyDelete
  3. MemoryStream has .WriteTo(outstream);

    and .NET 4.0 has .CopyTo on normal stream object.

    ReplyDelete
  4. There is actually, a less heavy-handed way of doing a stream copy. Take note however, that this implies that you can store the entire file in memory. Don't try and use this if you are working with files that go into the hundreds of megabytes or more, without caution.

    public static void CopyStream(Stream input, Stream output)
    {
    using (StreamReader reader = new StreamReader(input))
    using (StreamWriter writer = new StreamWriter(output))
    {
    writer.Write(reader.ReadToEnd());
    }
    }


    NOTE: There may also be some issues concerning binary data and character encodings.

    ReplyDelete
  5. The basic questions that differentiate implementations of "CopyStream" are:


    size of the reading buffer
    size of the writes
    Can we use more than one thread (writing while we are reading).


    The answers to these questions result in vastly different implementations of CopyStream and are dependent on what kind of streams you have and what you are trying to optimize. The "best" implementation would even need to know what specific hardware the streams were reading and writing to.

    ReplyDelete
  6. Unfortunately, there is no really simple solution. You can try something like that:

    Stream s1, s2;
    byte[] buffer = new byte[4096];
    int bytesRead = 0;
    while (bytesRead = s1.Read(buffer, 0, buffer.Length) > 0) s2.Write(buffer, 0, bytesRead);
    s1.Close(); s2.Close();


    But the problem with that that different implementation of the Stream class might behave differently if there is nothing to read. A stream reading a file from a local harddrive will probably block until the read operaition has read enough data from the disk to fill the buffer and only return less data if it reaches the end of file. On the other hand, a stream reading from the network might return less data even though there are more data left to be received.

    Always check the documentation of the specific stream class you are using before using a generic solution.

    ReplyDelete
  7. There may be a way to do this more efficiently, depending on what kind of stream you're working with. If you can convert one or both of your streams to a MemoryStream, you can use the GetBuffer method to work directly with a byte array representing your data. This lets you use methods like Array.CopyTo, which abstract away all the issues raised by fryguybob. You can just trust .NET to know the optimal way to copy the data.

    ReplyDelete
  8. if you want a procdure to copy a stream to other the one that nick posted is fine but it is missing the position reset, it should be

    public static void CopyStream(Stream input, Stream output)
    {
    byte[] buffer = new byte[32768];
    long TempPos = input.Position;
    while (true)
    {
    int read = input.Read (buffer, 0, buffer.Length);
    if (read <= 0)
    return;
    output.Write (buffer, 0, read);
    }
    input.Position = TempPos;// or you make Position = 0 to set it at the start
    }


    but if it is in runtime not using a procedure you shpuld use memory stream

    Stream output = new MemoryStream();
    byte[] buffer = new byte[32768]; // or you specify the size you want of your buffer
    long TempPos = input.Position;
    while (true)
    {
    int read = input.Read (buffer, 0, buffer.Length);
    if (read <= 0)
    return;
    output.Write (buffer, 0, read);
    }
    input.Position = TempPos;// or you make Position = 0 to set it at the start

    ReplyDelete
  9. public static void CopyStream(Stream input, Stream output)
    {
    byte[] buffer = new byte[32768];
    long TempPos = input.Position;
    while (true)
    {
    int read = input.Read (buffer, 0, buffer.Length);
    if (read <= 0) break;
    output.Write (buffer, 0, read);
    }
    input.Position = TempPos;// or you make Position = 0 to set it at the start
    }


    Use break instead of the return. The retrun prevents the input.Position = TempPos from being executed.

    ReplyDelete
  10. Since none of the answers have covered an asynchronous way of copying from one stream to another, here is a pattern that I've successfully used in a port forwarding application to copy data from one network stream to another. It lacks exception handling to emphasize the pattern.

    const int BUFFER_SIZE = 4096;

    static byte[] bufferForRead = new byte[BUFFER_SIZE];
    static byte[] bufferForWrite = new byte[BUFFER_SIZE];

    static Stream sourceStream = new MemoryStream();
    static Stream destinationStream = new MemoryStream();

    static void Main(string[] args)
    {
    // Initial read from source stream
    sourceStream.BeginRead(bufferForRead, 0, BUFFER_SIZE, BeginReadCallback, null);
    }

    private static void BeginReadCallback(IAsyncResult asyncRes)
    {
    // Finish reading from source stream
    int bytesRead = sourceStream.EndRead(asyncRes);
    // Make a copy of the buffer as we'll start another read immediately
    Array.Copy(bufferForRead, 0, bufferForWrite, 0, bytesRead);
    // Write copied buffer to destination stream
    destinationStream.BeginWrite(bufferForWrite, 0, bytesRead, BeginWriteCallback, null);
    // Start the next read (looks like async recursion I guess)
    sourceStream.BeginRead(bufferForRead, 0, BUFFER_SIZE, BeginReadCallback, null);
    }

    private static void BeginWriteCallback(IAsyncResult asyncRes)
    {
    // Finish writing to destination stream
    destinationStream.EndWrite(asyncRes);
    }

    ReplyDelete