If you return an R object from PL/R as bytea, it is passed through the R serialization interface.
This is mentioned in the fine manual, but not really prominently so.
I assume the rationale behind this is, that you can use R objects returned from one PL/R function directly as input to some other PL/R function.
But if you export an image or something else binary for use outside PL/R, you must get rid of the serialization first.
And there is a utility function for this in PL/R already:
Otherwise the binary output will not be what you expect...