Ruby TinyTds gem encoding error blocks all future access to database

Question

Project details:

• Ruby 1.8.7
• tiny_tds gem version 0.6.0.rc1
• connection to microsoft sql 2017
• ActiveRecord verson 2.3.14

The issue we're having is that we are trying to export about 1.5 million records with a rake task. I'm iterating over each record, then exporting it to json. Some of the data is 15 years old, and one table has some weird SQL_Latin1_General_CP1_CI_AS characters that break tiny_tds. What ends up happening though, is if I hit a record with the bad encoding (not many of them, out of 1.5 million I think it's only about 9,000), then every other iteration after that results in a strange ActiveRecord/TinyTds error.

It's a specific column. My solution was to catch the error, and report the id of the record to a file, here's the code:

association_names.to_a.each do |association_name|
  puts "Exporting '#{association_name}' from '#{record.class} - #{record.id}'"
  # associations = record.send(association_name).to_a
  associations = begin
    record.send(association_name).to_a
  # rescue TinyTds::Error => e
  rescue ActiveRecord::StatementInvalid => e
    puts "HIT THE TINYTDS ERROR"
    puts "#{e.class}: #{e.message}"
    puts e.backtrace.join("\n")
    File.open(File.join(pack_path, "overflow_errors.txt"), 'a') do |f|
      f.puts("Exporting '#{association_name}' from '#{record.class} - #{record.id}'")
    end
    []
  rescue StandardError => e
    puts "HIT THE STANDARD ERROR"
    puts "#{e.class}: #{e.message}"
    puts e.backtrace.join("\n")
    []
  end

  associations.each do |association_record|
    export_record(association_record, pack_path, scp)
  end
end

The place it throws the error is record.send(association_name). The first error is ActiveRecord::StatementInvalid: TinyTds::Error: Buffer overflow converting characters from client into server's character set. Which I catch, report, and skip to the next one. However, when this occurs, every following record (ones without the encoding issues, I've confirmed), I get ActiveRecord::StatementInvalid: NoMethodError: undefined method each' for false:FalseClass`. I was able to recreate the error in the console, you can see below:

>> Service.find(33333).service_details
=> []
>> ServiceData.find(444444).service_data # Record with bad encoding
ActiveRecord::StatementInvalid: TinyTds::Error: Buffer overflow converting characters from client into server's character set
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract_adapter.rb:174:in `log'
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:657:in `raw_select'
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:632:in `select'
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:489:in `find_by_sql'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1360:in `find_every'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1395:in `find_one'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1381:in `find_from_ids'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:437:in `find'
        from (irb):4
        from :0
>> Service.find(33333).service_details
/home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:659: warning: TinyTds: dbsqlsend() returned FAIL.

ActiveRecord::StatementInvalid: NoMethodError: undefined method `each' for false:FalseClass
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract_adapter.rb:174:in `log'
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:657:in `raw_select'
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/sqlserver_adapter.rb:632:in `select'
        from /home/my_user/the_project/lib/rails/002_active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:489:in `find_by_sql'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1360:in `find_every'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1395:in `find_one'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:1381:in `find_from_ids'
        from /home/my_user/the_project/lib/rails/002_active_record/zbase.rb:437:in `find'
        from (irb):5
        from :0

I cannot figure out why this is happening. When the TinyTds::Error occurs (which for some reason can only be caught with ActiveRecord::StatementInvalid), it seems to lock up the database. Also when I run the rake task, when the latter error occurs, I also get this warning: warning: TinyTds: dbsqlsend() returned FAIL..

I've dug into the activerecord code, and it does seem to return false sometimes, instead of an empty array. I tried changing it, but that lead to a bunch of other issues. However this error has never occurred in production, because we serve up this data with an asp server, which can handle this encoding. What could be causing this? Thanks.