How to use CP to copy variable length datasets and pds members?

Question

I've been trying to use cp under OMVS to copy z/OS datasets into OMVS files and the copy to OMVS seems to work but the copy back to z/OS fails miserably with multiple records combined.

I don't see any cp option to support variable length records.

I don't have dsfs available yet.

Help (please)

Answer 1

UNIX files are byte stream not record organized. Once you copy data into a UNIX file, all information on record boundaries is lost. Therefore, when you copy a UNIX file to some MVS data set, z/OS UNIX has no information on where to split the byte stream into records, except from the data set's logical record length (LRECL).

There is one exception: "text" files. In text files, lines are delimited by an end-of-line character or characters. When copying such a file to an MVS data set, the end-of-line character(s) are stripped, and a new record is started by each.

There are options to the cp command to tell is, what kind of data, and if applicable, whar kind of end-of-line character(s) to use on the UNIX file side.

Answer 2

I'm not sure if this helps or not since I'm not sure about your PDS members. However these examples work for me:

$ ls
$ cp "//'MYID.PROGRAMS.JCL(ASM2)'" .
$ ls
asm2

There is also a --sparse=always option for cp which might help.

Answer 3

Let's try some tests and assumptions. So I'll create dataset VB1.DATA as RECFM=VB LRECL=100 and add a few records like this:

           TED013.VB1.DATA                                 Columns 00001 00072 
Command ===>                                                  Scroll ===> CSR  
****** ***************************** Top of Data ******************************
000001 1111                                                                    
000002 22222222                                                                
000003 333333333333                                                            
****** **************************** Bottom of Data ****************************

Now I can use OMVS and copy that to a new USS file, and then examine the bytes:

cp "//vb1.data" vb1.data
od -x vb1.data                                           
0000000000      F1F1    F1F1    15F2    F2F2    F2F2    F2F2    F215    F3F3  
0000000020      F3F3    F3F3    F3F3    F3F3    F3F3    1500                  

You can see where cp added x'15' bytes to mark the ends of the variable length lines. And of course ISPF edit under ishell knows about the x'15' bytes and the USS file data looks just like the original source, despite the extra 15 at the end of the file (which is not the case in x86 Linux)

           /u/TED013/vb1.data                              Columns 00001 00072 
Command ===>                                                  Scroll ===> PAGE 
****** ***************************** Top of Data ******************************
000001 1111                                                                    
000002 22222222                                                                
000003 333333333333                                                            
****** **************************** Bottom of Data ****************************

So if I cp that back to MVS, a new VB dataset is created for me. The system decided on LRECL=1028 but that's no problem here.

cp vb1.data "//vb2.data" 

And that new VB2.DATA matches up exactly with the original VB1.DATA. So I don't see a problem. Yet...

So let's change one of the lines in the MVS source VB1.DATA and add a hex zero, such as line 2 below:

           TED013.VB1.DATA                                 Columns 00001 00072 
Command ===> hex on                                            Scroll ===> CSR  
****** ***************************** Top of Data ******************************
000001 1111                                                                    
       FFFF44444444444444444444444444444444444444444444444444444444444444444444
       111100000000000000000000000000000000000000000000000000000000000000000000
------------------------------------------------------------------------------ 
000002 2222 222                                                                
       FFFF0FFF4444444444444444444444444444444444444444444444444444444444444444
       222202220000000000000000000000000000000000000000000000000000000000000000
------------------------------------------------------------------------------ 
000003 333333333333                                                            
       FFFFFFFFFFFF444444444444444444444444444444444444444444444444444444444444
       333333333333000000000000000000000000000000000000000000000000000000000000
------------------------------------------------------------------------------ 
****** **************************** Bottom of Data ****************************

... and then cp and od to display:

cp "//vb1.data" vb1.data
od -x vb1.data                                         
0000000000      F1F1    F1F1    F2F2    F2F2    00F2    F2F2    F3F3    F3F3
0000000020      F3F3    F3F3    F3F3    F3F3                                

Wait a second, I see the zero, but what happened to our x'15' line breaks? The zero apparently causes cp to think the data is "binary" and eliminates the x'15' line breaks.

So let's copy back to MVS, and the result is a jumbled mess which I think matches Lionels description of "multiple records combined".

cp vb1.data "//vb2.data" 


           TED013.VB2.DATA                          Line 0000000000 Col 001 080 
 Command ===>                                                  Scroll ===> CSR  
********************************* Top of Data **********************************
11112222.222333333333333                                                        
******************************** Bottom of Data ********************************

So if my example matches Lionel's problem at all, my theory is:

1. Lionel's problem is not caused by the cp USS to MVS, but by the cp MVS to USS.
2. Lionel isn't using the -B, -X, -T, or -F parms on the cp command, which according to the book says "If you do not specify either -F|B|T or X, cp will first check the format of the MVS data set indicated and then try to determine the type of file."
3. That automated format decision seems to be based on if there is a x'00' somewhere in the file, at least in my quick tests.

So if I copy the file with the x'00' to USS and specify the -T option:

cp -T "//vb1.data" vb2.data 

od -x vb2.data                                          
0000000000      F1F1    F1F1    15F2    F2F2    F200    F2F2    F215    F3F3 
0000000020      F3F3    F3F3    F3F3    F3F3    F3F3    1500                 

... the x'15' bytes are back, and the x'00' is still there of course. So let's copy back to MVS using the same -T option so the x'15' bytes cause separate records:

cp -T vb2.data "//vb2.data"

           TED013.VB2.DATA                          Line 0000000000 Col 001 080 
 Command ===>                                                  Scroll ===> CSR  
********************************* Top of Data **********************************
1111                                                                            
2222.222                                                                        
333333333333                                                                    
******************************** Bottom of Data ********************************

And there's the data, as it started, including the x'00'

Now let's change the x'00' in the MVS source file to x'15', and run through the same process. We end up with:

           TED013.VB2.DATA                          Line 0000000000 Col 001 080 
 Command ===>                                                  Scroll ===> CSR  
********************************* Top of Data **********************************
1111                                                                            
2222                                                                            
222                                                                             
333333333333                                                                    
******************************** Bottom of Data ********************************

That's obviously a whole new problem, which (like phunsoft said) means binary and text can't always mix. But still, it looks like the designers of the cp logic between MVS and USS tried their best.