DBM file is readable by Perl scripts on the machine that made it, but "Inappropriate file type or format" on other machines

Question

I have a Perl script that makes a DBM index of a reference file using the DB_File module. Another Perl script then uses that DBM file. It works fine if I run both the setup and usage scripts on the same machine.

However, if I make the DBM file on machine A and copy it to machine B, the usage script on machine B can't use the DBM.

• Both machines have DB_File version 1.852.
• The DBM file is created with 0666 permissions, and sure enough, its permissions string from ls -ll is "-rw-r--r--".
• Machine A has Perl v5.26.2 and B has v5.18.4. Could this mismatch be the problem? B is a Mac and I've read that getting a newer version of Perl is not straightforward.

The reference file (names.txt):

2   |   Bacteria    |   Bacteria <bacteria> |   scientific name |
4640    |   Musa    |       |   scientific name |
9606    |   Homo sapiens    |       |   scientific name |

The setup script that makes the DBM:

#!/usr/bin/perl

    use strict;
    use warnings;
    use DB_File;
    use Fcntl;

my $namesfile = "names.txt";

my $namesfileDBMids = $namesfile . '_IDs.dbm';     
my %namesfileDBMids = (); # Start the hash that will fill the DBM file.
tie (%namesfileDBMids, "DB_File", $namesfileDBMids, O_RDWR|O_CREAT, 0666, $DB_HASH) or die "Can't open $namesfileDBMids.\n$!\n";

open (my $names_filehandle, $namesfile) or die "Could not open $namesfile.\n$!\n"; # Open the input file and fill the hash.
while (1) { # Run this loop until "last" is called.
            my $line = <$names_filehandle>; # Read the next line from the names file.

            if (! defined $line) { last }; # If there is no next line, exit the loop. You've processed the whole file.

            my @line = split(/\|/, $line); # Otherwise, split the line by | characters.
            my $name = $line[1];
            $name =~ s/^\s+|\s+$//g; # Trim whitespace off the start and end.
            my $ID = $line[0];
            $ID =~ s/^\s+|\s+$//g;
            $namesfileDBMids{$ID} = $name; # Store in the hash.
}

close $names_filehandle;
untie %namesfileDBMids;
print "Finished indexing.\n";  

And finally, this is the usage script that uses the DBM:

#!/usr/bin/perl

    use strict;
    use warnings;
    use DB_File;
    use Fcntl;

my $namesfileDBMids = "names.txt_IDs.dbm";

my $ID_to_look_up = 9606;

my %namesfileDBMids = (); # Set up a hash to hold the DBM file.
tie (%namesfileDBMids, "DB_File", $namesfileDBMids) or die "Can't open $namesfileDBMids: $!\n";

if (exists $namesfileDBMids{$ID_to_look_up}) {
    my $name = $namesfileDBMids{$ID_to_look_up};
    print "Found a name for ID $ID_to_look_up: $name\n";
} else {
    print "Couldn't find $ID_to_look_up in the names file.\n";
}

When the usage script can access the DBM file, it returns this line:

Found a name for ID 9606: Homo sapiens

When the usage script cannot access the DBM file, it either returns this (on machine B when the DBM was from A):

Can't open names.dmp_IDs.dbm: Inappropriate file type or format

Or this (on machine A when the DBM was from B):

Can't open names.dmp_IDs.dbm:

I'm not sure why there is no error message on A. It's a Linux server that I have limited access to.

Answer 1

It's very likely that your different versions of Perl were built using different versions of the DBM library that are incompatible at a binary level.

I would recommend switching to a text-based interchange format like YAML or JSON.

Answer 2

If you create .dbm files on each machine, the file command may reveal the version mismatch.

The problem could be anything from 32bit/64bit, the DB_File type, or hardware endianness.

Since you mentioned version 1.852 that means the module's been updated. v5.18.4 came with 1.827 and v5.26.2 came with 1.840. One thing to try is to go back to the core lib versions and make the DB file on 1.827 and then see if 1.840 can read it. And vice versa.

HTH