Skip to content

Commit 8a9f705

Browse files
committed
document producer pipe errors in v2 migration
Explain that go-pipe v1 could hide producer-side pipe write errors as an implementation detail of the command stdin-copy path, while v2 exposes those errors more directly. Document the migration patterns for stateless and stateful producers.
1 parent 6b61052 commit 8a9f705

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,31 @@ A package used to easily build command pipelines in your Go applications
44
# Important
55
We have not thoroughly tested this package on OSs other than Linux, especially Windows. At this time, using this package on Windows based systems is considered experimental and will be supported only on a best effort basis.
66

7+
# Migrating to v2
8+
9+
It's normal for pipelines to stop before all input has been consumed[^1]. If an earlier stage continues writing after that happens, the write side of the pipe can fail with `EPIPE`, `SIGPIPE`, or `io.ErrClosedPipe`.
10+
11+
In go-pipe v1 it was possible to get away without handling this case, because a command stage's stdin was connected in a way that often (but not necessarily!) drained the write side and hid the error from the previous stage feeding it. That was an implementation detail, not a guarantee. In go-pipe v2, producer stages are more likely to be connected directly to a command's stdin, and thus see the error themselves.
12+
13+
Fortunately, this is easily handled by wrapping the stage with `IgnoreError(stage, IsPipeError)`. If the producer only writes output and is otherwise stateless, that's the only thing needed.
14+
15+
If the producer also updates state, metrics, cursors, or has other side effects, in a way that depends on how much of the output was produced, then in addition to using `IgnoreError`, you must also ensure producer-owned state is brought to to a consistent point before returning the error.
16+
17+
For example, if a stateful producer function must process its entire input for correctness regardless of whether it was read by the consumer, it should use a pattern like:
18+
19+
```go
20+
var writeErr error
21+
for _, item := range items {
22+
updateState(item)
23+
if writeErr == nil {
24+
_, writeErr = fmt.Fprintln(stdout, item)
25+
}
26+
}
27+
return writeErr
28+
```
29+
730
# Links
831

932
* [Docs](https://pkg.go.dev/github.com/github/go-pipe/v2)
33+
34+
[^1]: In `cat foo | head | grep -q`, for example, either `head` or `grep` could exit before its input is fully consumed.

0 commit comments

Comments
 (0)