How does enumerators created with code block actually runs

Question

It's just a simple question, how is y.<< method is able to halt the code-block mid execution ??

I have expected the code block to run only once and never halt in the middle :/

e = Enumerator.new do |y|
    puts "Ruby"
    y << 1
    y << 2
    puts "Ruby"
    y << 3
end

puts e.each.next
puts e.each.next
puts e.each.next
e.rewind
puts e.each.next
puts e.each.next
puts e.each.next

Answer 1

Almost all Ruby implementations are Free Software and Open Source, so you can just look at the source code to see how it is implemented.

In Rubinius, the most interesting part is Enumerator::Iterator#reset, implemented in core/enumerator.rb:

@fiber = Fiber.new stack_size: STACK_SIZE do
  obj = @object
  @result = obj.each { |*val| Fiber.yield *val }
  @done = true
end

and Enumerator::Iterator#next:

val = @fiber.resume

TruffleRuby's implementation is very similar, as you can see in src/main/ruby/truffleruby/core/enumerator.rb:

class FiberGenerator
  # irrelevant methods omitted

  def next
    reset unless @fiber

    val = @fiber.resume

    raise StopIteration, 'iteration has ended' if @done

    val
  end

  def reset
    @done = false
    @fiber = Fiber.new do
      obj = @object
      @result = obj.each do |*val|
        Fiber.yield(*val)
      end
      @done = true
    end
  end
end

JRuby is also very similar, as you can see in core/src/main/ruby/jruby/kernel/enumerator.rb:

class FiberGenerator
  # irrelevant methods omitted

  def next
    reset unless @fiber&.__alive__

    val = @fiber.resume

    raise StopIteration, 'iteration has ended' if @state.done

    val
  end

  def reset
    @state.done = false
    @state.result = nil
    @fiber = Fiber.new(&@state)
  end

end

MRuby's implementation is very similar, as you can see in mrbgems/mruby-enumerator/mrblib/enumerator.rb.

YARV also uses Fibers, as can be seen in enumerator.c, for example here:

static void
next_init(VALUE obj, struct enumerator *e)
{
    VALUE curr = rb_fiber_current();
    e->dst = curr;
    e->fib = rb_fiber_new(next_i, obj);
    e->lookahead = Qundef;
}

static VALUE
get_next_values(VALUE obj, struct enumerator *e)
{
    VALUE curr, vs;

    if (e->stop_exc)
    rb_exc_raise(e->stop_exc);

    curr = rb_fiber_current();

    if (!e->fib || !rb_fiber_alive_p(e->fib)) {
    next_init(obj, e);
    }

    vs = rb_fiber_resume(e->fib, 1, &curr);
    if (e->stop_exc) {
    e->fib = 0;
    e->dst = Qnil;
    e->lookahead = Qundef;
    e->feedvalue = Qundef;
    rb_exc_raise(e->stop_exc);
    }
    return vs;
}

So, not surprisingly, Enumerator is implemented using Fibers in many Ruby implementations. Fiber is essentially just Ruby's name for semi-coroutines, and of course, coroutines are a popular way of implementing generators and iterators. E.g. CPython and CoreCLR also implement generators using coroutines.

One exception to this seems to be Opal. My assumption was that Opal would use ECMAScript Generators to implement Ruby Enumerators, but it does not look like that is the case. The implementation of Ruby Enumerators in Opal is found in opal/corelib/enumerator.rb, opal/corelib/enumerator/generator.rb, and opal/corelib/enumerator/yielder.rb with some help from opal/corelib/runtime.js, but unfortunately, I don't fully understand it. It does not appear to use either Ruby Fibers or ECMAScript Generators, though.

By the way, your usage of Enumerators is somewhat strange: you call Enumerator#each six times without a block, but calling Enumerator#each without a block just returns the Enumerator itself:

each → enum

Iterates over the block according to how this Enumerator was constructed. If no block and no arguments are given, returns self.

So, in other words, all those calls to Enumerator#each are just no-ops. It would make much more sense to just call Enumerator#next directly:

puts e.next
puts e.next
puts e.next
e.rewind
puts e.next
puts e.next
puts e.next