How can I upload data via HTTParty in Ruby while generating it?

Question

I have a function which generates some data. Right now, this data is written to a file. Once the file is complete, I upload it via HTTParty:

require 'httparty'

url = "..."

def generate_data(file)
  file << "First line of data\n"
  sleep 1
  file << "Second line of data\n"
  sleep 1
  file << "Third line of data\n"
end

File.open('payload.txt', 'w+') do |file|
  generate_data(file)
  file.rewind
  HTTParty.post(url, body: {file: file})
end

As it happens, generate_data takes a bit -- I would like to accelerate the script and avoid writing to disk by interleaving the generation of the data and uploading it. How could I do this using HTTParty?

I was looking for something like StringIO which could be used as a fixed-size FIFO buffer: the generate_data function writes to it (and blocks when the buffer is full) while the HTTParty.post call reads from it. (and blocks when the buffer is empty). However, I failed to find anything like that.

Answer 1

You need to use streaming

HTTParty.put(
  'http://localhost:3000/train',
  body_stream: StringIO.new('foo')
)